-- Logs begin at Tue 2022-08-02 08:06:16 UTC, end at Tue 2022-08-02 09:29:42 UTC. -- Aug 02 08:45:29.188119 dv-cinder-847131-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 02 08:45:29.199182 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 02 08:45:29.201201 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 2 08:45:29 2022] *** Aug 02 08:45:29.201201 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 02 08:45:29.201201 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 02 08:45:29.201201 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: nodename: dv-cinder-847131-4-iscsi Aug 02 08:45:29.201201 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: machine: x86_64 Aug 02 08:45:29.201201 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: clock source: unix Aug 02 08:45:29.201201 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: pcre jit disabled Aug 02 08:45:29.201201 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: detected number of CPU cores: 4 Aug 02 08:45:29.201201 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: current working directory: / Aug 02 08:45:29.201201 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: detected binary path: /usr/bin/uwsgi-core Aug 02 08:45:29.201747 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: your processes number limit is 47680 Aug 02 08:45:29.201747 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: your memory page size is 4096 bytes Aug 02 08:45:29.201747 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: detected max file descriptor number: 2048 Aug 02 08:45:29.201747 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: lock engine: pthread robust mutexes Aug 02 08:45:29.201747 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: thunder lock: enabled Aug 02 08:45:29.201747 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 02 08:45:29.201747 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 02 08:45:29.240663 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: Python main interpreter initialized at 0x5628d9a089c0 Aug 02 08:45:29.240663 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: python threads support enabled Aug 02 08:45:29.240663 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: your server socket listen backlog is limited to 100 connections Aug 02 08:45:29.240663 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: your mercy for graceful operations on workers is 90 seconds Aug 02 08:45:29.240993 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: mapped 403077 bytes (393 KB) for 2 cores Aug 02 08:45:29.241067 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: *** Operational MODE: preforking *** Aug 02 08:45:29.241335 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: *** uWSGI is running in multiple interpreter mode *** Aug 02 08:45:29.241335 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: spawned uWSGI master process (pid: 80316) Aug 02 08:45:29.241543 dv-cinder-847131-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 02 08:45:29.242340 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: spawned uWSGI worker 1 (pid: 80328, cores: 1) Aug 02 08:45:29.242445 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: spawned uWSGI worker 2 (pid: 80329, cores: 1) Aug 02 08:45:29.242707 dv-cinder-847131-4-iscsi devstack@keystone.service[80316]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 02 08:45:34.775110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 02 08:45:34.775110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: from cryptography.utils import int_from_bytes Aug 02 08:45:34.775918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 02 08:45:34.775918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: from cryptography.utils import int_from_bytes Aug 02 08:45:34.776326 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 02 08:45:34.776326 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: from cryptography.utils import int_from_bytes Aug 02 08:45:34.776787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 02 08:45:34.776787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: from cryptography.utils import int_from_bytes Aug 02 08:45:34.827580 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 02 08:45:34.827763 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 02 08:45:34.827965 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] command line args: [] {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 02 08:45:34.828244 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 02 08:45:34.828387 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 02 08:45:34.828473 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ================================================================================ {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 02 08:45:34.828589 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 02 08:45:34.828673 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] admin_token = **** {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.828790 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] command line args: [] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 02 08:45:34.828936 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] config_dir = [] {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.829067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 02 08:45:34.829145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.829254 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ================================================================================ {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 02 08:45:34.829344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] config_source = [] {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.829457 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] debug = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.829543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] admin_token = **** {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.829659 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.829989 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] config_dir = [] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.829989 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.830106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.830180 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] config_source = [] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.830245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.830307 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] debug = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.830473 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.830594 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.830733 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.830833 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.830929 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.831027 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] list_limit = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.831127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.831233 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] log_config_append = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.831338 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.831431 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.831535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] log_dir = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.831629 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] list_limit = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.831722 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] log_file = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.831820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] log_config_append = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.831925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.832003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.832122 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] log_dir = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.832201 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.832314 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] log_file = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.832407 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.832573 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.832651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.832718 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.832826 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.832903 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.832977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.833134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.833240 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.833370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.833450 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.833553 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.833668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.833802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.833936 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.844871 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.845067 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.845067 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.845267 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.845267 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.845398 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.845467 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.845540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.845622 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.845703 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.845779 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.845779 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.845938 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.845938 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] publish_errors = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.846141 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.846213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.846213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.846342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.846480 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.846553 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] publish_errors = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.846626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.846696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.846774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.846848 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.846848 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.847046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] standard_threads = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.847117 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.847196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.847266 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.847339 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.847469 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] standard_threads = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.847546 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.847546 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] use_journal = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.847677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.847746 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] use_json = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.847814 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.847882 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] use_stderr = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.847955 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.848024 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] use_syslog = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.848099 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] use_journal = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.848168 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.848241 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] use_json = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.848241 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] use_stderr = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.848368 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.848437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] use_syslog = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.848580 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.848651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 08:45:34.848719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.848792 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.848861 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.848934 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.849010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.849080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.849147 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.849220 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.849294 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.849368 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.849445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.849514 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.849581 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.849656 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.849749 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.849809 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.849956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.850032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.850101 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.850169 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.850236 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.850305 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.850380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.850451 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.850519 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.850587 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.850654 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.850733 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.850808 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.850878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.850946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.851020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.851095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.851175 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.851251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.851320 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.851394 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.851463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.851531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.851654 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.851735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.851853 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.851926 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.851995 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.852068 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.852211 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.852283 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.852357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.852426 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.852494 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.852561 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.852640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.852709 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.852783 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.852856 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.852947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.853016 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.853082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.853151 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.853266 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.853346 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.853416 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.853483 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.853559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.853634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.853709 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.853777 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.853845 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.853931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.854011 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.854088 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.854163 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.854232 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.854332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.854438 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.854541 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.854636 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.854730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.854829 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.854953 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.855096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.855210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.855315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.855392 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.855513 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.855588 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.855710 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.855796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.855872 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.855987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.856060 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.856173 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.856257 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.856336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.856412 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.856492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.856563 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.856677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.856677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.856852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.856934 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.857005 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.857005 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.857179 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.857179 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.857357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.857357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.857545 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.857671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.857785 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.857859 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.857958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.858085 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.858161 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.858161 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.858350 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.858496 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.858607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.858704 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.858819 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.858923 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.859000 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.859123 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.859198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.859320 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.859395 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.859472 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.859587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.859661 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.859774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.859855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.859926 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.860040 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.860113 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.860188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.860258 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.860335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.860405 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.860524 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.860605 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.860676 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.860744 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.860888 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.860961 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.861035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.861105 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.861173 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.861242 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.861318 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.861388 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.861457 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.861571 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.861645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.861714 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.861795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.861876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.861967 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.862042 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.862113 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.862182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.862251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.862338 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.862410 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.862479 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.862548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.862616 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.862685 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.862761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.862836 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.862897 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.862965 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.863033 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.863101 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.863176 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.863247 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.863315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.863383 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.863451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.863519 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.863594 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.863669 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.863739 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.863807 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.863881 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.863951 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.864026 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.864096 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.864164 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.864252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.864330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.864390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.864466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.864526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.864594 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.864662 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.864730 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.864804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.864880 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.864951 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.865025 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.865094 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.865180 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.865292 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.865379 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.865451 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.865520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] auth.external = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.865625 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.865700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.865770 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] auth.external = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.865847 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.865962 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.870196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.870318 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.870400 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] auth.password = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.870472 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.870542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] auth.token = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.870612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] auth.password = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.870690 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.870762 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] auth.token = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.870831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.870902 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.870971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.871041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.871117 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.871189 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.871259 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.871336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.871408 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.871485 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.871563 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.871641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.871712 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] credential.caching = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.871781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.871851 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.871928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] credential.caching = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.872006 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.872079 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.872148 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.872218 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.872287 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.872357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.872434 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.872505 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.872574 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.872643 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.872712 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.872789 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.872860 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.872929 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.872999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.873068 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.873138 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.873214 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.873214 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.873342 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.873342 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.873468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.873468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.873666 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.873666 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.873809 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.873942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.874020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.874090 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] federation.caching = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.874172 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.874245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.874316 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] federation.caching = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.874385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.874454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.874524 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.874643 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.874742 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.874836 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.874913 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.874985 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.875061 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.875140 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.875212 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.875282 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.875352 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.875422 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.875499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.875625 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.875702 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.875793 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.875871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.875943 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.876003 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] identity.caching = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.876164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.876238 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.876309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.caching = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.876379 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.876456 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.876535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.876615 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.876688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.876757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.876827 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.876897 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.876967 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.877045 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.877116 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.877186 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.877255 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.877332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.877403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.877480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.877552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.877621 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.877691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.877760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.877829 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.877931 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.878007 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.878077 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.878147 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.878217 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.878281 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.878359 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.878430 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.878501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.878570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.878640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.878718 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.878789 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.878859 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.878928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.878997 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.879066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.879147 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.879218 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.879289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.879358 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.879428 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.879498 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.879575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.879655 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.879727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.879803 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.879874 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.879944 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.880020 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.880092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.880161 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.880232 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.880338 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.880420 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.880591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.880672 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.880831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.880910 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.881082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.881170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.881332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.881411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.881565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.881649 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.881807 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.881923 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.882075 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.882156 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.882311 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.882389 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.882554 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.882632 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.882797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.882875 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.883035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.883112 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.883273 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.883351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.883513 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.883593 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.883764 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.883840 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.884067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.884663 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.884818 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.884898 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.885054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.885153 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.885310 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.885407 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.885562 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.885642 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.885802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.885897 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.886083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.886161 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.886322 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.886402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.user = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.886558 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.886636 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.886803 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.886883 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.887108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.887203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.887312 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.887436 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.887559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.887643 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.887809 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.887891 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.888052 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.888132 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.888287 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.888366 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.888517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.888596 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.888757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.888836 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.888994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.889073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.889230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.889311 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.889461 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.889539 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.889711 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.889791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.889981 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.890066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.890233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.890310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.890480 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.890595 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.890715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.890848 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.890966 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.891087 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.891207 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.891340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.891448 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.891583 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.891690 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.891823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.892048 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.892128 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.892290 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.892370 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.892538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.892617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.892780 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.892859 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.893019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.893099 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.893261 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.893339 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.893495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.893649 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.893732 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.893804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.893983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.894066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.894221 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.894312 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.894461 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.894541 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.894705 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.894783 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.894941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.895030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.895174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.895257 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.895414 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] resource.caching = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.895492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.895655 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] resource.caching = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.895741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.895907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.895986 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.896146 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.896225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.896387 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.896473 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.896638 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.896716 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.896876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.896962 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.897112 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.897198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.897353 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.897435 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.897592 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.897670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.897828 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] role.caching = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.897934 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.898093 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] role.caching = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.898172 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] role.driver = sql {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.898327 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.898406 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] role.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.898566 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.898652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.898819 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.898905 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.899068 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.899147 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.899316 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.899403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.899585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.899670 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.899839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.899917 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.900088 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.900176 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.900338 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.900418 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.900590 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.900668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.900824 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.900912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.901074 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.901153 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.901322 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.901400 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.901568 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.901659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.901813 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.902001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.902246 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.902326 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.902488 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.902568 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.902740 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.902819 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.902992 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.903072 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.903250 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.903333 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.903473 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.903559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.903713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.903791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.903970 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.904050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.904212 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.904292 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.904448 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.904527 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.904682 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.904762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.904914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.904993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.905147 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.905226 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.905402 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.905483 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.905651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.905730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.905905 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.905989 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.906151 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.906232 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.906384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.906464 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.906614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.906693 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] token.caching = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.906853 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.906933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] token.caching = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.907089 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.907167 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.907347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.907427 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.907569 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.907649 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.907829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.907909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.908069 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.908147 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.908310 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.908390 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.908558 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.908637 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.908805 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.908885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.909044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.909123 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.909286 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.909364 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.909522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.909601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.909757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.909836 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.910034 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.910114 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.910268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.910347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.910509 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.910589 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.910755 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.910835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.911047 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.911130 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] database.connection = **** {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.911294 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.911387 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.connection = **** {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.911546 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.911635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.911788 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.911869 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.912028 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.912109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.912266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.912346 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.912504 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.912583 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.912742 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.912820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.912964 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.913039 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.913207 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.913287 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.913445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.913523 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.913683 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.913763 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.913941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.914026 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.914182 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.914260 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.914422 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.914503 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.914663 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.914742 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.914897 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.914975 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.915134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.915214 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.915406 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.915487 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.915636 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80329) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 02 08:45:34.915715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 08:45:34.915795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80328) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 02 08:45:34.945589 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:34.945695 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:34.945831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:34.945931 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:34.946091 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:34.946178 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:34.955791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:34.956315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:34.986799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:34.986992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:34.987075 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:34.987151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.040222 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.040550 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.091469 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.091707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.092225 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.092466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.156893 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.157127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.157360 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.157588 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.157741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.157828 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.158010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.158110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.158244 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.158471 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.158697 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.158918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.181612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.181846 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.182119 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.182348 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.182509 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.182642 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.182752 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.182838 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.182986 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.183069 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.183214 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.183435 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.183652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.183870 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.194339 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.194920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.207950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.208189 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.208281 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.208417 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.208501 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.208646 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.208730 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.208870 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.208953 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.209093 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.209185 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.209315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.209397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80328) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.209533 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80329) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 08:45:35.227132 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.227344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 02 08:45:35.227496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.227707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.227790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 02 08:45:35.227914 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 02 08:45:35.228063 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.228207 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.228291 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 02 08:45:35.228454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.228613 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.228698 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.228877 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.228967 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.229136 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.229394 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.238741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.239025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.239120 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.239299 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.239382 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.239581 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.239663 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.239917 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.246633 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.247002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80329) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.251406 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.251698 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.251790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80329) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.251977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.252060 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.252263 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.252344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.252549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.252639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.252815 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.252896 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.253088 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.253169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.253366 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.253456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.253638 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.253717 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.253932 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.254014 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.254206 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.254289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.254476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.277698 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.278232 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.279221 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.279505 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.279738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80329) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.280021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.284144 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.284430 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.284662 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80329) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.284744 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.284945 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.285231 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.291957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.292196 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.292524 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80329) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.292809 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.296342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.296634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.296931 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.297055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80329) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.297217 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.297362 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.297495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.297646 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.297780 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.297949 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.298089 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.298231 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.298372 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.298520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.298792 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.299082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.314890 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.315169 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.315678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.315960 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.319131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.319905 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80329) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.323081 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.323380 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.323971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.324283 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.327822 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.328719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80329) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.331044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.331950 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80329) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.334235 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.334538 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.334833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.335128 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.335267 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.335420 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.335549 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.335709 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.335837 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.336051 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.336331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.336621 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.352324 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.352611 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.353397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.353664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.353934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80329) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.354223 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.354493 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.354759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.360638 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.361695 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.362221 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.362503 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.363274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.363552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.365328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.365614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.365940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.366230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.366384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80329) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.366508 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.366658 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.366958 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.367243 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.367538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.378639 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.378924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.379206 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.379487 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.379854 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80329) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.379979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.380105 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.380230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.380355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.380498 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.380626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.380753 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.381029 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.381304 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.395975 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.396994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80329) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.398985 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.400592 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80329) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.402475 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.403608 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.404973 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.406120 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.406570 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.406864 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.407152 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.407679 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80329) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.407966 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.408251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.413660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.414764 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.415850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.417009 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80329) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.418831 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.419128 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.419416 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.419706 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.419989 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.420285 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.420575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.420865 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.427821 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.428116 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.428400 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.428924 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80329) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.429222 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.429516 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.435325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80328) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.435607 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.435891 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.436170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.436428 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80329) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 08:45:35.436560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.436712 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.436868 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.436996 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.437135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.437263 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.437389 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.437514 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.437640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.437766 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.437930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.438059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.438196 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.438324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.438450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.438693 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.438828 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.438978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.439106 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80328) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.439273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.439549 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.439826 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.440089 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80329) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 08:45:35.472977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5628d9a089c0 pid: 80328 (default app) Aug 02 08:45:35.474225 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5628d9a089c0 pid: 80329 (default app) Aug 02 08:45:35.476269 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12aa4e-ec32-4f48-8484-f0417761bf85 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:35.476565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12aa4e-ec32-4f48-8484-f0417761bf85 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:35.476853 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f12aa4e-ec32-4f48-8484-f0417761bf85 None None] PATH_INFO: `/v3` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:35.481709 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1/1] 10.222.0.46 () {50 vars in 791 bytes} [Tue Aug 2 08:45:35 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:45:44.083717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078cf61-f152-4d62-a4ff-fc25d72b2566 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:44.084273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078cf61-f152-4d62-a4ff-fc25d72b2566 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:44.084368 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a078cf61-f152-4d62-a4ff-fc25d72b2566 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:44.088902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1/2] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:45:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:45:44.094316 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bb177-dfa5-4c42-9937-85805ee2a101 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:44.094714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bb177-dfa5-4c42-9937-85805ee2a101 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:44.094876 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2bb177-dfa5-4c42-9937-85805ee2a101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:44.166288 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: INFO dbcounter [None req-9d2bb177-dfa5-4c42-9937-85805ee2a101 None None] Registered counter for database keystone Aug 02 08:45:44.185939 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG oslo_db.sqlalchemy.engines [None req-9d2bb177-dfa5-4c42-9937-85805ee2a101 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80328) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 02 08:45:44.192701 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writer thread running {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 02 08:45:44.377105 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG passlib.handlers.bcrypt [None req-9d2bb177-dfa5-4c42-9937-85805ee2a101 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80328) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 02 08:45:44.377526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG passlib.handlers.bcrypt [None req-9d2bb177-dfa5-4c42-9937-85805ee2a101 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80328) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 02 08:45:44.431699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9d2bb177-dfa5-4c42-9937-85805ee2a101 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:45:44.433311 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9d2bb177-dfa5-4c42-9937-85805ee2a101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:45:44.562725 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 2/3] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:45:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 02 08:45:44.710564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078cf61-f152-4d62-a4ff-fc25d72b2566 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:44.781111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: INFO dbcounter [None req-a078cf61-f152-4d62-a4ff-fc25d72b2566 None None] Registered counter for database keystone Aug 02 08:45:44.802018 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG oslo_db.sqlalchemy.engines [None req-a078cf61-f152-4d62-a4ff-fc25d72b2566 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80329) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 02 08:45:44.808244 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writer thread running {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 02 08:45:44.929079 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a078cf61-f152-4d62-a4ff-fc25d72b2566 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:44.933467 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087cd29-477a-41f1-a2ab-2f869f4ddbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:44.934297 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087cd29-477a-41f1-a2ab-2f869f4ddbdd admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:44.934589 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087cd29-477a-41f1-a2ab-2f869f4ddbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:44.934866 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087cd29-477a-41f1-a2ab-2f869f4ddbdd admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:44.943277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087cd29-477a-41f1-a2ab-2f869f4ddbdd admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:44.995926 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5087cd29-477a-41f1-a2ab-2f869f4ddbdd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 02 08:45:44.996273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087cd29-477a-41f1-a2ab-2f869f4ddbdd admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:45.006291 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 2/4] 10.222.0.46 () {58 vars in 1324 bytes} [Tue Aug 2 08:45:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 08:45:45.012840 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2bb177-dfa5-4c42-9937-85805ee2a101 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:45.028407 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2bb177-dfa5-4c42-9937-85805ee2a101 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:45.032946 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49e91a4-f602-471f-8769-a4b7a2b31e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:45.034660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e91a4-f602-471f-8769-a4b7a2b31e34 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:45.034660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e91a4-f602-471f-8769-a4b7a2b31e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:45.034914 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49e91a4-f602-471f-8769-a4b7a2b31e34 admin admin] PATH_INFO: `/v3/projects/2a92b68d350c4f7998c221772fb56b4a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:45.036415 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e91a4-f602-471f-8769-a4b7a2b31e34 admin admin] RBAC: Authorizing `identity:get_project(project_id=2a92b68d350c4f7998c221772fb56b4a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:45.088667 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c49e91a4-f602-471f-8769-a4b7a2b31e34 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 02 08:45:45.089043 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49e91a4-f602-471f-8769-a4b7a2b31e34 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:45.093800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 3/5] 10.222.0.46 () {56 vars in 1207 bytes} [Tue Aug 2 08:45:45 2022] GET /identity/v3/projects/2a92b68d350c4f7998c221772fb56b4a => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:45:47.074053 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7883651-0482-4227-815f-00f6d54fff29 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:47.074598 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7883651-0482-4227-815f-00f6d54fff29 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:47.074598 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7883651-0482-4227-815f-00f6d54fff29 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:47.078655 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 3/6] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:45:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:45:47.083888 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e0828-b43a-4045-929b-16ef5e9bb954 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:47.084304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e0828-b43a-4045-929b-16ef5e9bb954 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:47.084454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e0828-b43a-4045-929b-16ef5e9bb954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:47.104769 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-2d9e0828-b43a-4045-929b-16ef5e9bb954 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:45:47.106322 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2d9e0828-b43a-4045-929b-16ef5e9bb954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:45:47.114056 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 4/7] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:45:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 02 08:45:47.136154 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7883651-0482-4227-815f-00f6d54fff29 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:47.148056 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7883651-0482-4227-815f-00f6d54fff29 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:47.153004 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2da71-e2a3-4e67-8307-63f6e9c858a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:47.153621 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2da71-e2a3-4e67-8307-63f6e9c858a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:47.153942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2da71-e2a3-4e67-8307-63f6e9c858a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:47.154230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2da71-e2a3-4e67-8307-63f6e9c858a8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:47.162131 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef2da71-e2a3-4e67-8307-63f6e9c858a8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:47.163925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef2da71-e2a3-4e67-8307-63f6e9c858a8 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:47.172810 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 4/8] 10.222.0.46 () {58 vars in 1324 bytes} [Tue Aug 2 08:45:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 08:45:47.178182 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e0828-b43a-4045-929b-16ef5e9bb954 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:47.187082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e0828-b43a-4045-929b-16ef5e9bb954 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:47.191480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed41ae3-2915-464b-9695-f5154e9ec2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:47.192058 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed41ae3-2915-464b-9695-f5154e9ec2ee admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:47.192341 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed41ae3-2915-464b-9695-f5154e9ec2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:47.192617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed41ae3-2915-464b-9695-f5154e9ec2ee admin admin] PATH_INFO: `/v3/users/5b34698a764b44aba3d550dca381a370` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:47.193998 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed41ae3-2915-464b-9695-f5154e9ec2ee admin admin] RBAC: Authorizing `identity:get_user(user_id=5b34698a764b44aba3d550dca381a370)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:47.195794 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed41ae3-2915-464b-9695-f5154e9ec2ee admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:47.198103 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 5/9] 10.222.0.46 () {56 vars in 1198 bytes} [Tue Aug 2 08:45:47 2022] GET /identity/v3/users/5b34698a764b44aba3d550dca381a370 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:45:49.261430 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80153e1-48d6-4526-becd-8ffe193629ca None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:49.261950 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80153e1-48d6-4526-becd-8ffe193629ca None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:49.261950 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80153e1-48d6-4526-becd-8ffe193629ca None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:49.263444 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 5/10] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:45:49.268216 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdb2d7-af20-4cdb-942b-dc2b6d448e19 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:49.268594 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdb2d7-af20-4cdb-942b-dc2b6d448e19 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:49.268911 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdb2d7-af20-4cdb-942b-dc2b6d448e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:49.288431 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f9cdb2d7-af20-4cdb-942b-dc2b6d448e19 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:45:49.289628 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f9cdb2d7-af20-4cdb-942b-dc2b6d448e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:45:49.312030 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097e0e4-fc23-4437-a4b2-c046280447c0 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:49.312400 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097e0e4-fc23-4437-a4b2-c046280447c0 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:49.312677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c097e0e4-fc23-4437-a4b2-c046280447c0 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:49.314018 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 6/11] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:45:49.318759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3dcad-df32-4baf-9b71-9ce3895dd8bc None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:49.319039 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3dcad-df32-4baf-9b71-9ce3895dd8bc None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:49.319314 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3dcad-df32-4baf-9b71-9ce3895dd8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:49.368927 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG passlib.handlers.bcrypt [None req-58f3dcad-df32-4baf-9b71-9ce3895dd8bc None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80329) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 02 08:45:49.369355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG passlib.handlers.bcrypt [None req-58f3dcad-df32-4baf-9b71-9ce3895dd8bc None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80329) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 02 08:45:49.373989 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 6/12] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:45:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:45:49.397133 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdb2d7-af20-4cdb-942b-dc2b6d448e19 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:49.415776 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-58f3dcad-df32-4baf-9b71-9ce3895dd8bc None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:45:49.417410 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-58f3dcad-df32-4baf-9b71-9ce3895dd8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:45:49.421715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:49.507658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdb2d7-af20-4cdb-942b-dc2b6d448e19 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:49.521953 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 7/13] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:45:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:45:49.541461 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd823308-f9d1-4c15-b912-d69eff6c091c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:49.542131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd823308-f9d1-4c15-b912-d69eff6c091c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:49.542413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd823308-f9d1-4c15-b912-d69eff6c091c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:49.542687 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd823308-f9d1-4c15-b912-d69eff6c091c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:49.544469 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3dcad-df32-4baf-9b71-9ce3895dd8bc None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:49.554708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd823308-f9d1-4c15-b912-d69eff6c091c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:45:49.563754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-fd823308-f9d1-4c15-b912-d69eff6c091c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:45:49.563754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:45:49.563754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:45:49.563754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:45:49.563754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:45:49.563754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:45:49.563754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:45:49.563754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:45:49.563754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:45:49.563754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:45:49.563754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:45:49.563754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:45:49.563754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:45:49.563754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:45:49.563754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:45:49.563754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:45:49.564730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:45:49.564730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:45:49.564730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:45:49.564730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:45:49.564730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:45:49.564730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:45:49.564730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:45:49.564730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:45:49.564730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:45:49.564730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:45:49.564730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:45:49.564730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:45:49.564730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:45:49.564730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:45:49.564730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:45:49.564730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:45:49.565550 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:45:49.565550 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:45:49.565550 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:45:49.565550 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:45:49.565550 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:45:49.565550 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:45:49.565550 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:45:49.565550 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:45:49.565550 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:45:49.565550 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:45:49.565550 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:45:49.565550 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:45:49.565550 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 7/14] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:45:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:45:49.570597 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd823308-f9d1-4c15-b912-d69eff6c091c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:49.645076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3dcad-df32-4baf-9b71-9ce3895dd8bc None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:49.647827 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:49.650942 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:49.669621 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd823308-f9d1-4c15-b912-d69eff6c091c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:49.683683 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b97bb-f788-4c19-a26d-37b2d19aa8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:49.684546 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b97bb-f788-4c19-a26d-37b2d19aa8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:49.684871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b97bb-f788-4c19-a26d-37b2d19aa8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:49.685151 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b97bb-f788-4c19-a26d-37b2d19aa8d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:49.704954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dc536-034b-4422-b7b6-a201961caa06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:49.705583 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dc536-034b-4422-b7b6-a201961caa06 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:49.705871 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dc536-034b-4422-b7b6-a201961caa06 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:49.706178 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01dc536-034b-4422-b7b6-a201961caa06 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:49.706898 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01dc536-034b-4422-b7b6-a201961caa06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:45:49.707205 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01dc536-034b-4422-b7b6-a201961caa06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:49.708607 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01dc536-034b-4422-b7b6-a201961caa06 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:49.721369 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 8/15] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:45:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:45:49.726926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dc536-034b-4422-b7b6-a201961caa06 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:49.755716 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b97bb-f788-4c19-a26d-37b2d19aa8d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:49.757218 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b97bb-f788-4c19-a26d-37b2d19aa8d7 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:49.826280 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01dc536-034b-4422-b7b6-a201961caa06 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:49.853911 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 8/16] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:45:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 08:45:49.859541 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b97bb-f788-4c19-a26d-37b2d19aa8d7 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:49.862699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:49.862827 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4467e2e-e4ea-49ed-aff3-981e29ad101c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:49.863457 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4467e2e-e4ea-49ed-aff3-981e29ad101c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:49.863745 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4467e2e-e4ea-49ed-aff3-981e29ad101c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:49.864026 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4467e2e-e4ea-49ed-aff3-981e29ad101c None admin] PATH_INFO: `/v3/users/5b34698a764b44aba3d550dca381a370` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:49.865221 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4467e2e-e4ea-49ed-aff3-981e29ad101c None admin] RBAC: Authorizing `identity:get_user(user_id=5b34698a764b44aba3d550dca381a370)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:49.866734 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4467e2e-e4ea-49ed-aff3-981e29ad101c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:49.868845 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 9/17] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:45:49 2022] GET /identity/v3/users/5b34698a764b44aba3d550dca381a370 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:45:49.874214 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4467e2e-e4ea-49ed-aff3-981e29ad101c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:49.889082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:49.960940 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b97bb-f788-4c19-a26d-37b2d19aa8d7 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:49.982841 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4467e2e-e4ea-49ed-aff3-981e29ad101c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:49.995052 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c60bb-c75a-4cf8-a0a0-3455345b4d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:49.995950 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c60bb-c75a-4cf8-a0a0-3455345b4d09 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:49.996238 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c60bb-c75a-4cf8-a0a0-3455345b4d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:49.996515 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7c60bb-c75a-4cf8-a0a0-3455345b4d09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:50.017577 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7c60bb-c75a-4cf8-a0a0-3455345b4d09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:50.019119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7c60bb-c75a-4cf8-a0a0-3455345b4d09 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:50.038934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-4a7c60bb-c75a-4cf8-a0a0-3455345b4d09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:45:50.038934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:45:50.038934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:45:50.038934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:45:50.038934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:45:50.038934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:45:50.038934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:45:50.038934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:45:50.038934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:45:50.038934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:45:50.038934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:45:50.038934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:45:50.038934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:45:50.038934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:45:50.038934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:45:50.040527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:45:50.040527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:45:50.040527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:45:50.040527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:45:50.040527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:45:50.040527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:45:50.040527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:45:50.040527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:45:50.040527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:45:50.040527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:45:50.040527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:45:50.040527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:45:50.040527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:45:50.040527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:45:50.040527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:45:50.040527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:45:50.040527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:45:50.041346 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:45:50.041346 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:45:50.041346 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:45:50.041346 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:45:50.041346 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 9/18] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:45:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:45:50.047216 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c60bb-c75a-4cf8-a0a0-3455345b4d09 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:50.107112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:50.125062 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfca8f-28e1-4299-a6fb-474769d3b1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:50.125716 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfca8f-28e1-4299-a6fb-474769d3b1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:50.126033 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfca8f-28e1-4299-a6fb-474769d3b1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:50.126320 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfca8f-28e1-4299-a6fb-474769d3b1c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:50.127531 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bfca8f-28e1-4299-a6fb-474769d3b1c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:50.128975 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bfca8f-28e1-4299-a6fb-474769d3b1c9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:50.130602 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 10/19] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:45:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:45:50.136543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfca8f-28e1-4299-a6fb-474769d3b1c9 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:50.151723 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7c60bb-c75a-4cf8-a0a0-3455345b4d09 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:50.185610 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4c4d4-217e-49fb-a55e-8415ae5c38cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:50.186243 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4c4d4-217e-49fb-a55e-8415ae5c38cb None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:50.186523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4c4d4-217e-49fb-a55e-8415ae5c38cb None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:50.186795 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4c4d4-217e-49fb-a55e-8415ae5c38cb None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:50.187481 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4c4d4-217e-49fb-a55e-8415ae5c38cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:45:50.187783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4c4d4-217e-49fb-a55e-8415ae5c38cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:50.189188 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d4c4d4-217e-49fb-a55e-8415ae5c38cb None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:50.206063 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 10/20] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:45:50.217607 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:50.239826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfca8f-28e1-4299-a6fb-474769d3b1c9 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:50.277792 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ebaa3-9cf7-4ce4-b493-f5c85bf17d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:50.278521 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ebaa3-9cf7-4ce4-b493-f5c85bf17d64 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:50.278732 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ebaa3-9cf7-4ce4-b493-f5c85bf17d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:50.279014 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ebaa3-9cf7-4ce4-b493-f5c85bf17d64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:50.279834 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ebaa3-9cf7-4ce4-b493-f5c85bf17d64 None admin] RBAC: Adding query filter params (role.id=8ce0850e15804f3d811e88cc9841e673, scope.domain.id=default, user.id=5b34698a764b44aba3d550dca381a370) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:45:50.280152 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ebaa3-9cf7-4ce4-b493-f5c85bf17d64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:50.281624 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ebaa3-9cf7-4ce4-b493-f5c85bf17d64 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:50.289170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 11/21] 10.222.0.46 () {56 vars in 1322 bytes} [Tue Aug 2 08:45:50 2022] GET /identity/v3/role_assignments?user.id=5b34698a764b44aba3d550dca381a370&scope.domain.id=default&role.id=8ce0850e15804f3d811e88cc9841e673 => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:45:52.489205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1e7a9-928d-4c2d-b747-f71efd57ec30 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:52.489760 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1e7a9-928d-4c2d-b747-f71efd57ec30 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:52.489760 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1e7a9-928d-4c2d-b747-f71efd57ec30 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:52.491118 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 11/22] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:45:52.507541 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8329853-d122-45a0-a752-1ca87935cfe2 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:52.508041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8329853-d122-45a0-a752-1ca87935cfe2 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:52.508117 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8329853-d122-45a0-a752-1ca87935cfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:52.534894 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-b8329853-d122-45a0-a752-1ca87935cfe2 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:45:52.536109 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b8329853-d122-45a0-a752-1ca87935cfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:45:52.608141 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 12/23] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:45:52.629916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1e7a9-928d-4c2d-b747-f71efd57ec30 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:52.747059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1e7a9-928d-4c2d-b747-f71efd57ec30 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:52.767968 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:52.772417 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63474da7-28cc-4c15-9371-ef7392c283f4 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:52.772705 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63474da7-28cc-4c15-9371-ef7392c283f4 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:52.772984 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63474da7-28cc-4c15-9371-ef7392c283f4 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:52.774464 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 13/24] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:45:52.779983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9514bb-da1a-43e8-9140-19fc111dab6c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:52.780265 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9514bb-da1a-43e8-9140-19fc111dab6c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:52.780541 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9514bb-da1a-43e8-9140-19fc111dab6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:52.797166 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eaa6b2-4c44-42c6-9334-31d8d136470a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:52.798081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eaa6b2-4c44-42c6-9334-31d8d136470a None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:52.798258 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eaa6b2-4c44-42c6-9334-31d8d136470a None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:52.798407 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eaa6b2-4c44-42c6-9334-31d8d136470a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:52.806826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-ed9514bb-da1a-43e8-9140-19fc111dab6c None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:45:52.808091 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ed9514bb-da1a-43e8-9140-19fc111dab6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:45:52.819039 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eaa6b2-4c44-42c6-9334-31d8d136470a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:45:52.828835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-d2eaa6b2-4c44-42c6-9334-31d8d136470a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:45:52.828835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:45:52.828835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:45:52.828835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:45:52.828835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:45:52.828835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:45:52.828835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:45:52.828835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:45:52.828835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:45:52.828835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:45:52.828835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:45:52.828835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:45:52.828835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:45:52.828835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:45:52.828835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:45:52.828835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:45:52.831184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:45:52.831184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:45:52.831184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:45:52.831184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:45:52.831184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:45:52.831184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:45:52.831184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:45:52.831184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:45:52.831184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:45:52.831184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:45:52.831184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:45:52.831184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:45:52.831184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:45:52.831184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:45:52.831184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:45:52.831184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:45:52.833100 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:45:52.833100 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:45:52.833100 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:45:52.833100 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:45:52.833100 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:45:52.833100 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:45:52.833100 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:45:52.833100 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:45:52.833100 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:45:52.833100 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:45:52.833100 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:45:52.833100 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:45:52.833100 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 12/25] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:45:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:45:52.840510 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eaa6b2-4c44-42c6-9334-31d8d136470a None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:52.966229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 14/26] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 02 08:45:53.040501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b53ad0-8026-432c-acbf-535db5723b5d None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:53.040788 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b53ad0-8026-432c-acbf-535db5723b5d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:53.040980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b53ad0-8026-432c-acbf-535db5723b5d None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:53.042583 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 15/27] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:45:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:45:53.048755 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d49ab-bf1c-4bde-bb17-4e7d487dbf89 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:53.049044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d49ab-bf1c-4bde-bb17-4e7d487dbf89 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:53.049321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d49ab-bf1c-4bde-bb17-4e7d487dbf89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:53.070257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:53.080149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-a28d49ab-bf1c-4bde-bb17-4e7d487dbf89 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:45:53.081396 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a28d49ab-bf1c-4bde-bb17-4e7d487dbf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:45:53.110058 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eaa6b2-4c44-42c6-9334-31d8d136470a None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:53.153641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9f19b-92c0-4441-a582-026903e8b77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:53.154296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9f19b-92c0-4441-a582-026903e8b77d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:53.155387 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9f19b-92c0-4441-a582-026903e8b77d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:53.155667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9f19b-92c0-4441-a582-026903e8b77d None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:53.156398 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9f19b-92c0-4441-a582-026903e8b77d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:45:53.156707 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9f19b-92c0-4441-a582-026903e8b77d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:53.158126 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9f19b-92c0-4441-a582-026903e8b77d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:53.172095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 13/28] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:45:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:45:53.177270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9f19b-92c0-4441-a582-026903e8b77d None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:53.196626 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 16/29] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:45:53.198579 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4d7da-b5c4-47f8-b612-93ad647b666c None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:53.198876 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4d7da-b5c4-47f8-b612-93ad647b666c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:53.199160 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4d7da-b5c4-47f8-b612-93ad647b666c None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:53.204608 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 17/30] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:45:53.204608 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4d7da-b5c4-47f8-b612-93ad647b666c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:53.242521 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:53.326070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9f19b-92c0-4441-a582-026903e8b77d None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:53.335079 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4d7da-b5c4-47f8-b612-93ad647b666c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:53.371211 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0251e7-f11d-452e-85e4-33c4df71ba82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:53.371828 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0251e7-f11d-452e-85e4-33c4df71ba82 None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:53.372313 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0251e7-f11d-452e-85e4-33c4df71ba82 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:53.372788 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0251e7-f11d-452e-85e4-33c4df71ba82 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:53.373671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0251e7-f11d-452e-85e4-33c4df71ba82 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:53.375324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0251e7-f11d-452e-85e4-33c4df71ba82 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:53.379273 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4f8e9-988c-4083-8c54-7c5e852ee2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:53.380032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4f8e9-988c-4083-8c54-7c5e852ee2cc None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:53.380466 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4f8e9-988c-4083-8c54-7c5e852ee2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:53.380933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4f8e9-988c-4083-8c54-7c5e852ee2cc None admin] PATH_INFO: `/v3/users/5b34698a764b44aba3d550dca381a370` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:53.382298 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4f8e9-988c-4083-8c54-7c5e852ee2cc None admin] RBAC: Authorizing `identity:get_user(user_id=5b34698a764b44aba3d550dca381a370)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:53.384764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4f8e9-988c-4083-8c54-7c5e852ee2cc None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:53.386798 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 18/31] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:45:53 2022] GET /identity/v3/users/5b34698a764b44aba3d550dca381a370 => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:45:53.388225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be27bd-080a-4d38-b3cc-9fd940c11d8a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:53.388748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be27bd-080a-4d38-b3cc-9fd940c11d8a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:53.389191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be27bd-080a-4d38-b3cc-9fd940c11d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:53.396428 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:53.410269 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f1be27bd-080a-4d38-b3cc-9fd940c11d8a None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:45:53.411710 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f1be27bd-080a-4d38-b3cc-9fd940c11d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:45:53.478639 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 19/32] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:45:53.479486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be27bd-080a-4d38-b3cc-9fd940c11d8a None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:53.608723 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be27bd-080a-4d38-b3cc-9fd940c11d8a None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:53.633999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 14/33] 10.222.0.46 () {60 vars in 1131 bytes} [Tue Aug 2 08:45:53 2022] POST /identity/v3/roles => generated 219 bytes in 458 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:45:53.635160 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0251e7-f11d-452e-85e4-33c4df71ba82 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:53.654034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:53.665991 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac531d19-a90b-4bd3-986e-aa34198a0bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:53.666700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac531d19-a90b-4bd3-986e-aa34198a0bba None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:53.666991 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac531d19-a90b-4bd3-986e-aa34198a0bba None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:53.667273 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac531d19-a90b-4bd3-986e-aa34198a0bba None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:53.668346 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac531d19-a90b-4bd3-986e-aa34198a0bba None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:53.669796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac531d19-a90b-4bd3-986e-aa34198a0bba None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:53.779456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:53.785296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0251e7-f11d-452e-85e4-33c4df71ba82 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:53.825189 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677e1a0-46f7-4253-9126-a2437165601f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:53.826312 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677e1a0-46f7-4253-9126-a2437165601f None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:53.826791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677e1a0-46f7-4253-9126-a2437165601f None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:53.827230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677e1a0-46f7-4253-9126-a2437165601f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:53.829058 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8677e1a0-46f7-4253-9126-a2437165601f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:53.830882 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8677e1a0-46f7-4253-9126-a2437165601f None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:53.833156 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 15/34] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:45:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:45:53.834337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677e1a0-46f7-4253-9126-a2437165601f None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:53.963601 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677e1a0-46f7-4253-9126-a2437165601f None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:53.973121 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 20/35] 10.222.0.46 () {60 vars in 1131 bytes} [Tue Aug 2 08:45:53 2022] POST /identity/v3/roles => generated 225 bytes in 494 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:45:53.977971 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac531d19-a90b-4bd3-986e-aa34198a0bba None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:54.017745 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12856854-c6ea-4334-9db5-514ac121900b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:54.018935 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12856854-c6ea-4334-9db5-514ac121900b None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:54.019418 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12856854-c6ea-4334-9db5-514ac121900b None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:54.019885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12856854-c6ea-4334-9db5-514ac121900b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:54.032818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12856854-c6ea-4334-9db5-514ac121900b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:54.034903 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12856854-c6ea-4334-9db5-514ac121900b None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:54.052585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-12856854-c6ea-4334-9db5-514ac121900b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:45:54.052585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:45:54.052585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:45:54.052585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:45:54.052585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:45:54.052585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:45:54.052585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:45:54.052585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:45:54.052585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:45:54.052585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:45:54.052585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:45:54.052585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:45:54.052585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:45:54.052585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:45:54.052585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:45:54.053476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:45:54.053476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:45:54.053476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:45:54.053476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:45:54.053476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:45:54.053476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:45:54.053476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:45:54.053476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:45:54.053476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:45:54.053476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:45:54.053476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:45:54.053476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:45:54.053476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:45:54.053476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:45:54.053476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:45:54.053476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:45:54.053476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:45:54.054764 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:45:54.054764 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:45:54.054764 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:45:54.054764 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:45:54.057023 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 16/36] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:45:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:45:54.059091 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a937ee7-f9a8-43a2-8f44-6264840e0af0 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:54.059663 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a937ee7-f9a8-43a2-8f44-6264840e0af0 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:54.060151 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a937ee7-f9a8-43a2-8f44-6264840e0af0 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:54.062267 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 17/37] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:45:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:45:54.063394 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a937ee7-f9a8-43a2-8f44-6264840e0af0 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:54.086099 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:54.125806 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac531d19-a90b-4bd3-986e-aa34198a0bba None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:54.175350 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6bac7-aead-4164-978f-1b22c62df4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:54.176043 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6bac7-aead-4164-978f-1b22c62df4d0 None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:54.176332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6bac7-aead-4164-978f-1b22c62df4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:54.176608 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6bac7-aead-4164-978f-1b22c62df4d0 None admin] PATH_INFO: `/v3/domains/default/users/5b34698a764b44aba3d550dca381a370/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:54.178817 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6bac7-aead-4164-978f-1b22c62df4d0 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5b34698a764b44aba3d550dca381a370, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:54.180990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6bac7-aead-4164-978f-1b22c62df4d0 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:54.190428 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a937ee7-f9a8-43a2-8f44-6264840e0af0 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:54.227036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae9042a-e8db-4967-8f56-bbc16121f491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:54.227680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9042a-e8db-4967-8f56-bbc16121f491 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:54.227970 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9042a-e8db-4967-8f56-bbc16121f491 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:54.228245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae9042a-e8db-4967-8f56-bbc16121f491 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:54.228938 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9042a-e8db-4967-8f56-bbc16121f491 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:45:54.229244 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9042a-e8db-4967-8f56-bbc16121f491 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:54.230748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae9042a-e8db-4967-8f56-bbc16121f491 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:54.242014 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 18/38] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:45:54.243860 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c034480-e03c-4687-99ec-c737a0b338c8 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:54.244194 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c034480-e03c-4687-99ec-c737a0b338c8 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:54.244421 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c034480-e03c-4687-99ec-c737a0b338c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:54.263149 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-3c034480-e03c-4687-99ec-c737a0b338c8 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:45:54.264671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-3c034480-e03c-4687-99ec-c737a0b338c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:45:54.274060 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 21/39] 10.222.0.46 () {58 vars in 1361 bytes} [Tue Aug 2 08:45:53 2022] PUT /identity/v3/domains/default/users/5b34698a764b44aba3d550dca381a370/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:45:54.275529 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90b57f-8f67-48e8-92e4-59e4cc56452c None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:54.275886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90b57f-8f67-48e8-92e4-59e4cc56452c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:54.276239 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90b57f-8f67-48e8-92e4-59e4cc56452c None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:54.277645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 22/40] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:45:54.278481 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90b57f-8f67-48e8-92e4-59e4cc56452c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:54.344979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:54.355996 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 19/41] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:45:54.357510 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a1164-7cc0-49ff-8a1b-e419266b2336 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:54.357809 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a1164-7cc0-49ff-8a1b-e419266b2336 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:54.358128 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a1164-7cc0-49ff-8a1b-e419266b2336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:54.389682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-2d8a1164-7cc0-49ff-8a1b-e419266b2336 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:45:54.390967 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-2d8a1164-7cc0-49ff-8a1b-e419266b2336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:45:54.393496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:54.406621 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90b57f-8f67-48e8-92e4-59e4cc56452c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:54.456337 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b205f-59b7-4def-b3ab-07f5a6bbd4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:54.456883 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b205f-59b7-4def-b3ab-07f5a6bbd4bd None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:54.457170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b205f-59b7-4def-b3ab-07f5a6bbd4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:54.457451 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b205f-59b7-4def-b3ab-07f5a6bbd4bd None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:54.458678 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b205f-59b7-4def-b3ab-07f5a6bbd4bd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:54.460150 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b205f-59b7-4def-b3ab-07f5a6bbd4bd None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:54.472078 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 20/42] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:45:54.472804 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a1164-7cc0-49ff-8a1b-e419266b2336 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:54.576905 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a1164-7cc0-49ff-8a1b-e419266b2336 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:54.598088 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 23/43] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 2 08:45:54 2022] POST /identity/v3/projects => generated 301 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:45:54.598929 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b205f-59b7-4def-b3ab-07f5a6bbd4bd None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:54.616881 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d5e38-d70c-41b3-8b84-cc795716ccc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:54.619534 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d5e38-d70c-41b3-8b84-cc795716ccc2 None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:54.620082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d5e38-d70c-41b3-8b84-cc795716ccc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:54.620889 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d5e38-d70c-41b3-8b84-cc795716ccc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:54.622229 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d5e38-d70c-41b3-8b84-cc795716ccc2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:54.624415 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d5e38-d70c-41b3-8b84-cc795716ccc2 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:54.703618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:54.714072 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 21/44] 10.222.0.46 () {60 vars in 1131 bytes} [Tue Aug 2 08:45:54 2022] POST /identity/v3/roles => generated 223 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:45:54.726314 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b205f-59b7-4def-b3ab-07f5a6bbd4bd None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:54.730942 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:54.773092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c652e25-8faa-4b66-8a4e-392ccb105ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:54.773771 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c652e25-8faa-4b66-8a4e-392ccb105ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:54.774126 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c652e25-8faa-4b66-8a4e-392ccb105ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:54.774410 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c652e25-8faa-4b66-8a4e-392ccb105ec8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:54.775804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c652e25-8faa-4b66-8a4e-392ccb105ec8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:54.777415 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c652e25-8faa-4b66-8a4e-392ccb105ec8 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:54.779341 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 24/45] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:45:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:45:54.786446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d5e38-d70c-41b3-8b84-cc795716ccc2 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:54.905982 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d5e38-d70c-41b3-8b84-cc795716ccc2 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:54.947666 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cc65d9-8605-41f6-997c-7ec234621f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:54.948306 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc65d9-8605-41f6-997c-7ec234621f5b None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:54.948595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc65d9-8605-41f6-997c-7ec234621f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:54.948873 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cc65d9-8605-41f6-997c-7ec234621f5b None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:54.949570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc65d9-8605-41f6-997c-7ec234621f5b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:54.951063 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cc65d9-8605-41f6-997c-7ec234621f5b None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:55.061760 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 22/46] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 2 08:45:54 2022] POST /identity/v3/projects => generated 312 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:45:56.338943 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6210b-cb1d-4157-935f-7cc25971834f None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:56.339437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6210b-cb1d-4157-935f-7cc25971834f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:56.339437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6210b-cb1d-4157-935f-7cc25971834f None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:56.340806 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 25/47] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:45:56.346008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049296c2-87d7-4126-9158-44f3b57e686f None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:56.346398 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049296c2-87d7-4126-9158-44f3b57e686f None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:56.346576 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049296c2-87d7-4126-9158-44f3b57e686f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:56.365027 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-049296c2-87d7-4126-9158-44f3b57e686f None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:45:56.366238 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-049296c2-87d7-4126-9158-44f3b57e686f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:45:56.413066 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:56.433346 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 23/48] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:45:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:45:56.456378 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6210b-cb1d-4157-935f-7cc25971834f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:56.557587 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6210b-cb1d-4157-935f-7cc25971834f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:56.591487 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf2662-d0c1-4009-b013-75dd95847e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:56.592069 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf2662-d0c1-4009-b013-75dd95847e0b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:56.592354 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf2662-d0c1-4009-b013-75dd95847e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:56.592632 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf2662-d0c1-4009-b013-75dd95847e0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:56.603197 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf2662-d0c1-4009-b013-75dd95847e0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:45:56.608148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-69bf2662-d0c1-4009-b013-75dd95847e0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:45:56.608148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:45:56.608148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:45:56.608148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:45:56.608148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:45:56.608148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:45:56.608148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:45:56.608148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:45:56.608148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:45:56.608148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:45:56.608148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:45:56.608148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:45:56.608148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:45:56.608148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:45:56.608148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:45:56.608148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:45:56.609110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:45:56.609110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:45:56.609110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:45:56.609110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:45:56.609110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:45:56.609110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:45:56.609110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:45:56.609110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:45:56.609110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:45:56.609110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:45:56.609110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:45:56.609110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:45:56.609110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:45:56.609110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:45:56.609110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:45:56.609110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:45:56.610083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:45:56.610083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:45:56.610083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:45:56.610083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:45:56.610083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:45:56.610083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:45:56.610083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:45:56.610083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:45:56.610083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:45:56.610083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:45:56.610083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:45:56.610083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:45:56.610083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 26/49] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:45:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:45:56.615120 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049296c2-87d7-4126-9158-44f3b57e686f None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:56.713032 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049296c2-87d7-4126-9158-44f3b57e686f None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:56.746917 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad53d1a-5ca7-41ff-9746-9d76e4256c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:56.747542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad53d1a-5ca7-41ff-9746-9d76e4256c9f None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:56.747827 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad53d1a-5ca7-41ff-9746-9d76e4256c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:56.748104 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad53d1a-5ca7-41ff-9746-9d76e4256c9f None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:56.748824 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad53d1a-5ca7-41ff-9746-9d76e4256c9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:45:56.749132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad53d1a-5ca7-41ff-9746-9d76e4256c9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:56.750555 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad53d1a-5ca7-41ff-9746-9d76e4256c9f None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:56.759291 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 24/50] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:45:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:45:56.764574 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf2662-d0c1-4009-b013-75dd95847e0b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:56.796018 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:56.865523 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf2662-d0c1-4009-b013-75dd95847e0b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:56.898976 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f2ec5-04f0-476e-8f33-5d67d8b7f1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:56.900120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f2ec5-04f0-476e-8f33-5d67d8b7f1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:56.900407 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f2ec5-04f0-476e-8f33-5d67d8b7f1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:56.900690 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f2ec5-04f0-476e-8f33-5d67d8b7f1e9 None admin] PATH_INFO: `/v3/users/5b34698a764b44aba3d550dca381a370` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:56.901858 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f2ec5-04f0-476e-8f33-5d67d8b7f1e9 None admin] RBAC: Authorizing `identity:get_user(user_id=5b34698a764b44aba3d550dca381a370)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:56.903353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f2ec5-04f0-476e-8f33-5d67d8b7f1e9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:56.904967 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 27/51] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:45:56 2022] GET /identity/v3/users/5b34698a764b44aba3d550dca381a370 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:45:56.910425 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad53d1a-5ca7-41ff-9746-9d76e4256c9f None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:56.954138 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:57.010628 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad53d1a-5ca7-41ff-9746-9d76e4256c9f None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:57.043577 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a614f-6315-42ca-9182-de8ef02e7df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:57.044226 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a614f-6315-42ca-9182-de8ef02e7df2 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:57.044512 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a614f-6315-42ca-9182-de8ef02e7df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:57.044788 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a614f-6315-42ca-9182-de8ef02e7df2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:57.045969 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a614f-6315-42ca-9182-de8ef02e7df2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:57.047411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a614f-6315-42ca-9182-de8ef02e7df2 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:57.048646 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f36e07-e2f9-43c9-89bd-878a2a1c1b57 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:57.048940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f36e07-e2f9-43c9-89bd-878a2a1c1b57 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:57.049162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 25/52] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:45:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:45:57.049249 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f36e07-e2f9-43c9-89bd-878a2a1c1b57 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:57.050583 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 28/53] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:45:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:45:57.054442 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a614f-6315-42ca-9182-de8ef02e7df2 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:57.055447 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d671f-2d3e-4376-b5bf-d750a698148e None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:57.055731 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d671f-2d3e-4376-b5bf-d750a698148e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:57.056012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d671f-2d3e-4376-b5bf-d750a698148e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:57.074657 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f27d671f-2d3e-4376-b5bf-d750a698148e None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:45:57.076186 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f27d671f-2d3e-4376-b5bf-d750a698148e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:45:57.142322 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 29/54] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:45:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:45:57.155562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a614f-6315-42ca-9182-de8ef02e7df2 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:57.165377 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d671f-2d3e-4376-b5bf-d750a698148e None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:57.166082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:57.188994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f24b8-336d-4f1e-9518-67e90c5c2ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:57.189613 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f24b8-336d-4f1e-9518-67e90c5c2ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:57.189945 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f24b8-336d-4f1e-9518-67e90c5c2ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:57.190239 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f24b8-336d-4f1e-9518-67e90c5c2ed7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:57.191019 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f24b8-336d-4f1e-9518-67e90c5c2ed7 None admin] RBAC: Adding query filter params (role.id=8ce0850e15804f3d811e88cc9841e673, scope.domain.id=default, user.id=5b34698a764b44aba3d550dca381a370) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:45:57.191338 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f24b8-336d-4f1e-9518-67e90c5c2ed7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:57.192765 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f24b8-336d-4f1e-9518-67e90c5c2ed7 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:57.193680 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:57.202320 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 26/55] 10.222.0.46 () {56 vars in 1322 bytes} [Tue Aug 2 08:45:57 2022] GET /identity/v3/role_assignments?user.id=5b34698a764b44aba3d550dca381a370&scope.domain.id=default&role.id=8ce0850e15804f3d811e88cc9841e673 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:45:57.268303 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d671f-2d3e-4376-b5bf-d750a698148e None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:57.304060 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6904b-035a-4eae-bf88-c25cc6e81e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:57.304733 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6904b-035a-4eae-bf88-c25cc6e81e58 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:57.305012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6904b-035a-4eae-bf88-c25cc6e81e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:57.305825 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6904b-035a-4eae-bf88-c25cc6e81e58 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:57.307051 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6904b-035a-4eae-bf88-c25cc6e81e58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:57.308535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6904b-035a-4eae-bf88-c25cc6e81e58 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:57.310227 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 30/56] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:45:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:45:57.315994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f24b8-336d-4f1e-9518-67e90c5c2ed7 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:57.418721 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f24b8-336d-4f1e-9518-67e90c5c2ed7 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:57.452112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7edbb-db2c-4fb9-a629-db1f313f1bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:57.452722 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7edbb-db2c-4fb9-a629-db1f313f1bd2 None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:57.453006 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7edbb-db2c-4fb9-a629-db1f313f1bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:57.453284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7edbb-db2c-4fb9-a629-db1f313f1bd2 None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:57.454004 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c7edbb-db2c-4fb9-a629-db1f313f1bd2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:57.455437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c7edbb-db2c-4fb9-a629-db1f313f1bd2 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:57.574591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 27/57] 10.222.0.46 () {60 vars in 1140 bytes} [Tue Aug 2 08:45:57 2022] POST /identity/v3/projects => generated 298 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:45:59.539832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9694a-415f-4fcd-bbfd-5088f56d1697 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:59.540348 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9694a-415f-4fcd-bbfd-5088f56d1697 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:59.540432 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab9694a-415f-4fcd-bbfd-5088f56d1697 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:59.541756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 31/58] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:45:59.546737 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0669c2-4331-47a0-bb46-64120aa43ef0 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:59.547129 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0669c2-4331-47a0-bb46-64120aa43ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:59.547298 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0669c2-4331-47a0-bb46-64120aa43ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:59.565264 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-2c0669c2-4331-47a0-bb46-64120aa43ef0 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:45:59.566473 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-2c0669c2-4331-47a0-bb46-64120aa43ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:45:59.587077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:59.632407 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 28/59] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:45:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:45:59.657333 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9694a-415f-4fcd-bbfd-5088f56d1697 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:59.751104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:45:59.755591 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab9694a-415f-4fcd-bbfd-5088f56d1697 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:59.788177 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3851cfb4-37a2-4e15-88d2-8fea34ad0a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:59.788815 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851cfb4-37a2-4e15-88d2-8fea34ad0a54 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:59.789104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851cfb4-37a2-4e15-88d2-8fea34ad0a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:59.789943 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851cfb4-37a2-4e15-88d2-8fea34ad0a54 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:59.791070 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3851cfb4-37a2-4e15-88d2-8fea34ad0a54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:59.792502 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3851cfb4-37a2-4e15-88d2-8fea34ad0a54 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:45:59.794119 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 32/60] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:45:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:45:59.799738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0669c2-4331-47a0-bb46-64120aa43ef0 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:45:59.896400 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0669c2-4331-47a0-bb46-64120aa43ef0 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:45:59.929966 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083aa504-8aff-46ea-ac2e-83d43127a48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:45:59.931202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083aa504-8aff-46ea-ac2e-83d43127a48f None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:45:59.931488 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083aa504-8aff-46ea-ac2e-83d43127a48f None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:45:59.931769 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083aa504-8aff-46ea-ac2e-83d43127a48f None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:45:59.932452 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083aa504-8aff-46ea-ac2e-83d43127a48f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:45:59.934009 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083aa504-8aff-46ea-ac2e-83d43127a48f None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:00.044909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 29/61] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 2 08:45:59 2022] POST /identity/v3/users => generated 275 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:46:03.191190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6fdd1-e6c6-4f28-b0c8-55c6f82381a0 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:03.191670 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6fdd1-e6c6-4f28-b0c8-55c6f82381a0 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:03.191670 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c6fdd1-e6c6-4f28-b0c8-55c6f82381a0 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:03.193018 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 33/62] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:03.202200 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ff6ff-e2ce-4e7b-a7e8-af36dfd53d8a None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:03.203112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ff6ff-e2ce-4e7b-a7e8-af36dfd53d8a None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:03.203614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ff6ff-e2ce-4e7b-a7e8-af36dfd53d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:03.226021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-365ff6ff-e2ce-4e7b-a7e8-af36dfd53d8a None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:03.227896 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-365ff6ff-e2ce-4e7b-a7e8-af36dfd53d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:03.240103 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:03.313983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 30/63] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:46:03.347031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c6fdd1-e6c6-4f28-b0c8-55c6f82381a0 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:03.385562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89d7a6-99bf-4e65-8004-c4cbb35d9eea None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:03.386108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89d7a6-99bf-4e65-8004-c4cbb35d9eea None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:03.386557 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89d7a6-99bf-4e65-8004-c4cbb35d9eea None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:03.388267 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 31/64] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:46:03.393211 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4742d-018f-41ae-86e6-d355bb4019c4 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:03.393693 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4742d-018f-41ae-86e6-d355bb4019c4 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:03.394150 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e4742d-018f-41ae-86e6-d355bb4019c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:03.423779 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-81e4742d-018f-41ae-86e6-d355bb4019c4 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:03.425828 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-81e4742d-018f-41ae-86e6-d355bb4019c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:03.544047 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c6fdd1-e6c6-4f28-b0c8-55c6f82381a0 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:03.548431 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 32/65] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:46:03.550009 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efcd00-69b9-4601-81ad-66d1cb76b1a2 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:03.550464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efcd00-69b9-4601-81ad-66d1cb76b1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:03.550945 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efcd00-69b9-4601-81ad-66d1cb76b1a2 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:03.552327 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 33/66] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:03.561175 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6143a0a-9681-47d8-89de-e5ccca3ee9a7 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:03.561673 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6143a0a-9681-47d8-89de-e5ccca3ee9a7 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:03.562119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6143a0a-9681-47d8-89de-e5ccca3ee9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:03.582529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-a6143a0a-9681-47d8-89de-e5ccca3ee9a7 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:03.584223 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-a6143a0a-9681-47d8-89de-e5ccca3ee9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:03.602340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10567f-718b-488f-bd1d-c942a7a9b5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:03.603114 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10567f-718b-488f-bd1d-c942a7a9b5e5 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:03.603395 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10567f-718b-488f-bd1d-c942a7a9b5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:03.603668 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb10567f-718b-488f-bd1d-c942a7a9b5e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:03.614944 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb10567f-718b-488f-bd1d-c942a7a9b5e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:03.620518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-bb10567f-718b-488f-bd1d-c942a7a9b5e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:03.620518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:03.620518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:03.620518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:03.620518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:03.620518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:03.620518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:03.620518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:03.620518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:03.620518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:03.620518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:03.620518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:03.620518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:03.620518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:03.620518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:03.620518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:03.621381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:03.621381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:03.621381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:03.621381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:03.621381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:03.621381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:03.621381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:03.621381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:03.621381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:03.621381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:03.621381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:03.621381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:03.621381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:03.621381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:03.621381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:03.621381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:03.622186 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:03.622186 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:03.622186 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:03.622186 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:03.622186 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:03.622186 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:03.622186 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:03.622186 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:03.622186 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:03.622186 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:03.622186 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:03.622186 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:03.622186 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 34/67] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:03.622822 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10567f-718b-488f-bd1d-c942a7a9b5e5 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:03.654392 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:03.661437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 34/68] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:03.662721 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba6b1a-6546-4ab1-9ddb-f7e3bd5abe96 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:03.663018 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba6b1a-6546-4ab1-9ddb-f7e3bd5abe96 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:03.672936 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ba6b1a-6546-4ab1-9ddb-f7e3bd5abe96 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:03.672936 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 35/69] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:03.672936 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba6b1a-6546-4ab1-9ddb-f7e3bd5abe96 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:03.672936 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:03.732523 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb10567f-718b-488f-bd1d-c942a7a9b5e5 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:03.766095 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9370544-ac17-4da5-af96-de047d3b50cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:03.766972 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9370544-ac17-4da5-af96-de047d3b50cf None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:03.767260 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9370544-ac17-4da5-af96-de047d3b50cf None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:03.767543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9370544-ac17-4da5-af96-de047d3b50cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:03.770640 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ba6b1a-6546-4ab1-9ddb-f7e3bd5abe96 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:03.778213 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9370544-ac17-4da5-af96-de047d3b50cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:03.783340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-e9370544-ac17-4da5-af96-de047d3b50cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:46:03.783340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:03.783340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:03.783340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:03.783340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:03.783340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:03.783340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:03.783340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:03.783340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:03.783340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:03.783340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:03.783340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:03.783340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:03.783340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:03.783340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:03.783340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:03.789977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:03.789977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:03.789977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:03.789977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:03.789977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:03.789977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:03.789977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:03.789977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:03.789977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:03.789977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:03.789977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:03.789977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:03.789977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:03.789977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:03.789977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:03.789977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:03.791031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:03.791031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:03.791031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:03.791031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:03.791031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:03.791031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:03.791031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:03.791031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:03.791031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:03.791031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:03.791031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:46:03.791031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:03.791031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 35/70] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:46:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:03.791031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4216bf4-a393-4aa8-b7e4-ae929a372eeb None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:03.791705 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4216bf4-a393-4aa8-b7e4-ae929a372eeb None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:03.791705 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4216bf4-a393-4aa8-b7e4-ae929a372eeb None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:03.791705 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 36/71] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:03.793124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3efe36-95c2-4371-a2aa-8784df0b9157 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:03.793410 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3efe36-95c2-4371-a2aa-8784df0b9157 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:03.793691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3efe36-95c2-4371-a2aa-8784df0b9157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:03.809940 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c47cb-63a2-4666-a4d8-5383e7d6d6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:03.810531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c47cb-63a2-4666-a4d8-5383e7d6d6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:03.810819 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c47cb-63a2-4666-a4d8-5383e7d6d6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:03.811102 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c47cb-63a2-4666-a4d8-5383e7d6d6d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:03.811576 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-6c3efe36-95c2-4371-a2aa-8784df0b9157 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:03.811817 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c47cb-63a2-4666-a4d8-5383e7d6d6d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:03.812130 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c47cb-63a2-4666-a4d8-5383e7d6d6d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:03.812791 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6c3efe36-95c2-4371-a2aa-8784df0b9157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:03.813518 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c47cb-63a2-4666-a4d8-5383e7d6d6d8 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:03.821629 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 36/72] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:03.822706 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c47cb-63a2-4666-a4d8-5383e7d6d6d8 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:03.883006 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 37/73] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:03.884276 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c77cf9-f1fe-4e81-a889-4a25e70dca2c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:03.884577 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c77cf9-f1fe-4e81-a889-4a25e70dca2c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:03.884862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c77cf9-f1fe-4e81-a889-4a25e70dca2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:03.896789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:03.902484 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-e7c77cf9-f1fe-4e81-a889-4a25e70dca2c None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:03.903670 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e7c77cf9-f1fe-4e81-a889-4a25e70dca2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:03.915567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:03.937132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c47cb-63a2-4666-a4d8-5383e7d6d6d8 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:03.972220 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eedab7-48b9-4e31-baa8-07e01276e841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:03.972867 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eedab7-48b9-4e31-baa8-07e01276e841 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:03.972986 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 38/74] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:03.973753 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c77cf9-f1fe-4e81-a889-4a25e70dca2c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:03.974130 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eedab7-48b9-4e31-baa8-07e01276e841 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:03.974446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eedab7-48b9-4e31-baa8-07e01276e841 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:03.988918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eedab7-48b9-4e31-baa8-07e01276e841 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:03.996271 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-f9eedab7-48b9-4e31-baa8-07e01276e841 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:03.996271 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:03.996271 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:03.996271 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:03.996271 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:03.996271 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:03.996271 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:03.996271 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:03.996271 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:03.996271 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:03.996271 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:03.996271 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:03.996271 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:03.996271 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:03.996271 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:03.996271 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:03.997134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:03.997134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:03.997134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:03.997134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:03.997134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:03.997134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:03.997134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:03.997134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:03.997134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:03.997134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:03.997134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:03.997134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:03.997134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:03.997134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:03.997134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:03.997134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:03.997930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:03.997930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:03.997930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:03.997930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:03.997930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:03.997930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:03.997930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:03.997930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:03.997930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:03.997930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:03.997930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:03.997930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:03.997930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 37/75] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 2 08:46:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:03.998602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eedab7-48b9-4e31-baa8-07e01276e841 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:04.080052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c77cf9-f1fe-4e81-a889-4a25e70dca2c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:04.102996 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eedab7-48b9-4e31-baa8-07e01276e841 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:04.118239 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49109248-1af1-4e37-91fe-f8999753d7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:04.118861 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49109248-1af1-4e37-91fe-f8999753d7e7 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:04.119149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49109248-1af1-4e37-91fe-f8999753d7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:04.119429 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49109248-1af1-4e37-91fe-f8999753d7e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:04.120151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49109248-1af1-4e37-91fe-f8999753d7e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:04.120458 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49109248-1af1-4e37-91fe-f8999753d7e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:04.121852 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49109248-1af1-4e37-91fe-f8999753d7e7 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:04.126841 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:04.130194 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 39/76] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:04.131002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49109248-1af1-4e37-91fe-f8999753d7e7 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:04.141498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacad16f-0ae6-44ee-a0bc-2e090d0f1c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:04.141498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacad16f-0ae6-44ee-a0bc-2e090d0f1c0f None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:04.141498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacad16f-0ae6-44ee-a0bc-2e090d0f1c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:04.141498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacad16f-0ae6-44ee-a0bc-2e090d0f1c0f None admin] PATH_INFO: `/v3/users/7e1257a546984dca904f536574103621` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:04.147221 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:04.157260 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacad16f-0ae6-44ee-a0bc-2e090d0f1c0f None admin] RBAC: Authorizing `identity:get_user(user_id=7e1257a546984dca904f536574103621)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:04.158764 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacad16f-0ae6-44ee-a0bc-2e090d0f1c0f None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:04.160455 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 38/77] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:03 2022] GET /identity/v3/users/7e1257a546984dca904f536574103621 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:04.161360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacad16f-0ae6-44ee-a0bc-2e090d0f1c0f None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:04.239322 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49109248-1af1-4e37-91fe-f8999753d7e7 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:04.266065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacad16f-0ae6-44ee-a0bc-2e090d0f1c0f None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:04.273410 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd84143-1616-456d-994b-2cc7eb656ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:04.274610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd84143-1616-456d-994b-2cc7eb656ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:04.274893 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd84143-1616-456d-994b-2cc7eb656ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:04.275171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd84143-1616-456d-994b-2cc7eb656ca2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:04.285031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd84143-1616-456d-994b-2cc7eb656ca2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:04.290296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-ddd84143-1616-456d-994b-2cc7eb656ca2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:04.290296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:04.290296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:04.290296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:04.290296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:04.290296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:04.290296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:04.290296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:04.290296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:04.290296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:04.290296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:04.290296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:04.290296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:04.290296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:04.290296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:04.290296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:04.291137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:04.291137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:04.291137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:04.291137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:04.291137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:04.291137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:04.291137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:04.291137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:04.291137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:04.291137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:04.291137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:04.291137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:04.291137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:04.291137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:04.291137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:04.291137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:04.292309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:04.292309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:04.292309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:04.292309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:04.292309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:04.292309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:04.292309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:04.292309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:04.292309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:04.292309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:04.292309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:04.292309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:04.292309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 40/78] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:04.292309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd84143-1616-456d-994b-2cc7eb656ca2 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:04.303920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47061355-e1ca-44a9-aee0-6e0aecf9fc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:04.304574 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47061355-e1ca-44a9-aee0-6e0aecf9fc5b None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:04.304862 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47061355-e1ca-44a9-aee0-6e0aecf9fc5b None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:04.305168 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47061355-e1ca-44a9-aee0-6e0aecf9fc5b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:04.306364 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47061355-e1ca-44a9-aee0-6e0aecf9fc5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:04.307820 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47061355-e1ca-44a9-aee0-6e0aecf9fc5b None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:04.309403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 39/79] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:46:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:04.309403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47061355-e1ca-44a9-aee0-6e0aecf9fc5b None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:04.359144 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:04.359301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:04.391726 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd84143-1616-456d-994b-2cc7eb656ca2 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:04.407946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47061355-e1ca-44a9-aee0-6e0aecf9fc5b None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:04.424236 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded6550-5374-4592-b272-dda744049bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:04.425045 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded6550-5374-4592-b272-dda744049bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:04.425491 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded6550-5374-4592-b272-dda744049bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:04.425821 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded6550-5374-4592-b272-dda744049bf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:04.426988 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded6550-5374-4592-b272-dda744049bf9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:04.427299 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded6550-5374-4592-b272-dda744049bf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:04.428688 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded6550-5374-4592-b272-dda744049bf9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:04.437036 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 41/80] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:46:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:04.442312 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded6550-5374-4592-b272-dda744049bf9 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:04.442761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf807d46-1285-42d9-ac97-2c031e6fbbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:04.443401 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf807d46-1285-42d9-ac97-2c031e6fbbb1 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:04.443683 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf807d46-1285-42d9-ac97-2c031e6fbbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:04.443962 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf807d46-1285-42d9-ac97-2c031e6fbbb1 None admin] PATH_INFO: `/v3/users/5b34698a764b44aba3d550dca381a370` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:04.445182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf807d46-1285-42d9-ac97-2c031e6fbbb1 None admin] RBAC: Authorizing `identity:get_user(user_id=5b34698a764b44aba3d550dca381a370)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:04.446754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf807d46-1285-42d9-ac97-2c031e6fbbb1 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:04.448641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 40/81] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:04 2022] GET /identity/v3/users/5b34698a764b44aba3d550dca381a370 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:04.449402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf807d46-1285-42d9-ac97-2c031e6fbbb1 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:04.544464 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded6550-5374-4592-b272-dda744049bf9 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:04.560736 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf807d46-1285-42d9-ac97-2c031e6fbbb1 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:04.573010 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:04.578012 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:04.579199 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeabd8d-f82f-45fb-aca4-75a25d53bae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:04.584379 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeabd8d-f82f-45fb-aca4-75a25d53bae7 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:04.584379 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeabd8d-f82f-45fb-aca4-75a25d53bae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:04.584379 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeabd8d-f82f-45fb-aca4-75a25d53bae7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:04.595174 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231c464-b7dd-40d8-b103-510886880fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:04.595797 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231c464-b7dd-40d8-b103-510886880fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:04.596084 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231c464-b7dd-40d8-b103-510886880fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:04.596364 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231c464-b7dd-40d8-b103-510886880fd2 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:04.597077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231c464-b7dd-40d8-b103-510886880fd2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:04.597387 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231c464-b7dd-40d8-b103-510886880fd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:04.598810 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231c464-b7dd-40d8-b103-510886880fd2 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:04.607939 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 41/82] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:04.608723 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231c464-b7dd-40d8-b103-510886880fd2 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:04.649865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeabd8d-f82f-45fb-aca4-75a25d53bae7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:04.651327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebeabd8d-f82f-45fb-aca4-75a25d53bae7 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:04.706752 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231c464-b7dd-40d8-b103-510886880fd2 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:04.741694 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a6bf0-9e2d-4969-ac62-640d678e6997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:04.742344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a6bf0-9e2d-4969-ac62-640d678e6997 None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:04.743143 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a6bf0-9e2d-4969-ac62-640d678e6997 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:04.743422 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a6bf0-9e2d-4969-ac62-640d678e6997 None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:04.744105 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a6bf0-9e2d-4969-ac62-640d678e6997 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:04.745566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a6bf0-9e2d-4969-ac62-640d678e6997 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:04.748761 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 42/83] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:46:04.749603 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeabd8d-f82f-45fb-aca4-75a25d53bae7 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:04.781944 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:04.853815 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeabd8d-f82f-45fb-aca4-75a25d53bae7 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:04.879077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 42/84] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 2 08:46:04 2022] POST /identity/v3/users => generated 289 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:46:04.879983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a6bf0-9e2d-4969-ac62-640d678e6997 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:04.895796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d021200-07a4-4483-9ec1-4e6ccf00c520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:04.896484 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d021200-07a4-4483-9ec1-4e6ccf00c520 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:04.899368 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d021200-07a4-4483-9ec1-4e6ccf00c520 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:04.899368 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d021200-07a4-4483-9ec1-4e6ccf00c520 None admin] PATH_INFO: `/v3/users/7e1257a546984dca904f536574103621` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:04.899368 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d021200-07a4-4483-9ec1-4e6ccf00c520 None admin] RBAC: Authorizing `identity:get_user(user_id=7e1257a546984dca904f536574103621)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:04.901101 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d021200-07a4-4483-9ec1-4e6ccf00c520 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:04.907926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 43/85] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:04 2022] GET /identity/v3/users/7e1257a546984dca904f536574103621 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:04.907926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d021200-07a4-4483-9ec1-4e6ccf00c520 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:04.936123 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:05.010124 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a6bf0-9e2d-4969-ac62-640d678e6997 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:05.020927 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:05.026834 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d021200-07a4-4483-9ec1-4e6ccf00c520 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:05.056761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43768bc-26c7-462f-b4ea-748e9168c696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:05.057443 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43768bc-26c7-462f-b4ea-748e9168c696 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:05.058019 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43768bc-26c7-462f-b4ea-748e9168c696 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:05.058019 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43768bc-26c7-462f-b4ea-748e9168c696 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:05.068158 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80f52f-99d5-46b5-bbff-030c61b2a9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:05.068816 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80f52f-99d5-46b5-bbff-030c61b2a9fe None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:05.069098 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80f52f-99d5-46b5-bbff-030c61b2a9fe None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:05.069376 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80f52f-99d5-46b5-bbff-030c61b2a9fe None admin] PATH_INFO: `/v3/users/7e1257a546984dca904f536574103621` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:05.070660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80f52f-99d5-46b5-bbff-030c61b2a9fe None admin] RBAC: Authorizing `identity:get_user(user_id=7e1257a546984dca904f536574103621)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:05.072557 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80f52f-99d5-46b5-bbff-030c61b2a9fe None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:05.074437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 44/86] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:04 2022] GET /identity/v3/users/7e1257a546984dca904f536574103621 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:05.075607 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80f52f-99d5-46b5-bbff-030c61b2a9fe None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:05.153838 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43768bc-26c7-462f-b4ea-748e9168c696 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:05.156092 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43768bc-26c7-462f-b4ea-748e9168c696 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:05.202182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:05.207970 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80f52f-99d5-46b5-bbff-030c61b2a9fe None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:05.255559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1b870-a12e-4327-9908-bd82a4857190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:05.257161 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1b870-a12e-4327-9908-bd82a4857190 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:05.257594 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1b870-a12e-4327-9908-bd82a4857190 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:05.258825 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c1b870-a12e-4327-9908-bd82a4857190 None admin] PATH_INFO: `/v3/projects/592a7a9f7334413584fa4b0d080c1f29` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:05.260676 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c1b870-a12e-4327-9908-bd82a4857190 None admin] RBAC: Authorizing `identity:get_project(project_id=592a7a9f7334413584fa4b0d080c1f29)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:05.262614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c1b870-a12e-4327-9908-bd82a4857190 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:05.265172 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 45/87] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:05 2022] GET /identity/v3/projects/592a7a9f7334413584fa4b0d080c1f29 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:05.266155 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1b870-a12e-4327-9908-bd82a4857190 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:05.288270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 43/88] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:46:05.290390 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43768bc-26c7-462f-b4ea-748e9168c696 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:05.309099 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:05.384151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c1b870-a12e-4327-9908-bd82a4857190 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:05.412817 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43768bc-26c7-462f-b4ea-748e9168c696 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:05.423356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb952d-6289-4889-a24a-586998d69151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:05.424129 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb952d-6289-4889-a24a-586998d69151 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:05.424834 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb952d-6289-4889-a24a-586998d69151 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:05.425104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fb952d-6289-4889-a24a-586998d69151 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:05.449930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19881f8-3039-420c-8270-9d62ab36cc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:05.450786 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19881f8-3039-420c-8270-9d62ab36cc0f None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:05.451305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19881f8-3039-420c-8270-9d62ab36cc0f None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:05.451943 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19881f8-3039-420c-8270-9d62ab36cc0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:05.456317 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:05.503811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb952d-6289-4889-a24a-586998d69151 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:05.505338 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fb952d-6289-4889-a24a-586998d69151 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:05.526851 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19881f8-3039-420c-8270-9d62ab36cc0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:05.528694 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19881f8-3039-420c-8270-9d62ab36cc0f None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:05.543907 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:05.620612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 46/89] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:46:05.621476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb952d-6289-4889-a24a-586998d69151 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:05.633353 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 44/90] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:46:05.634606 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19881f8-3039-420c-8270-9d62ab36cc0f None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:05.670625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:05.733875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fb952d-6289-4889-a24a-586998d69151 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:05.738118 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19881f8-3039-420c-8270-9d62ab36cc0f None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:05.775576 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb19f6-9fe8-49ec-a95d-b08b5c98b41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:05.776122 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb19f6-9fe8-49ec-a95d-b08b5c98b41d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:05.776412 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb19f6-9fe8-49ec-a95d-b08b5c98b41d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:05.776703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb19f6-9fe8-49ec-a95d-b08b5c98b41d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:05.777523 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb19f6-9fe8-49ec-a95d-b08b5c98b41d None admin] RBAC: Adding query filter params (role.id=68fe920c9be84b3aa1d35bdce554d96a, scope.project.id=592a7a9f7334413584fa4b0d080c1f29, user.id=7e1257a546984dca904f536574103621) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:05.777832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb19f6-9fe8-49ec-a95d-b08b5c98b41d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:05.779540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb19f6-9fe8-49ec-a95d-b08b5c98b41d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:05.781629 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:05.786721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 47/91] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:46:05 2022] GET /identity/v3/role_assignments?user.id=7e1257a546984dca904f536574103621&scope.project.id=592a7a9f7334413584fa4b0d080c1f29&role.id=68fe920c9be84b3aa1d35bdce554d96a => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:05.787212 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c393c-6101-4324-96c1-b55e1848badf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:05.787511 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb19f6-9fe8-49ec-a95d-b08b5c98b41d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:05.788293 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c393c-6101-4324-96c1-b55e1848badf None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:05.788838 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c393c-6101-4324-96c1-b55e1848badf None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:05.789340 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c393c-6101-4324-96c1-b55e1848badf None admin] PATH_INFO: `/v3/projects/592a7a9f7334413584fa4b0d080c1f29` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:05.795324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c393c-6101-4324-96c1-b55e1848badf None admin] RBAC: Authorizing `identity:get_project(project_id=592a7a9f7334413584fa4b0d080c1f29)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:05.799665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c393c-6101-4324-96c1-b55e1848badf None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:05.803680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 45/92] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:05 2022] GET /identity/v3/projects/592a7a9f7334413584fa4b0d080c1f29 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:05.810244 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c393c-6101-4324-96c1-b55e1848badf None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:05.950172 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb19f6-9fe8-49ec-a95d-b08b5c98b41d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:06.001348 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86565264-78eb-4ab4-919d-3df4dfdc2711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:06.002042 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86565264-78eb-4ab4-919d-3df4dfdc2711 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:06.002336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86565264-78eb-4ab4-919d-3df4dfdc2711 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:06.002674 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86565264-78eb-4ab4-919d-3df4dfdc2711 None admin] PATH_INFO: `/v3/projects/592a7a9f7334413584fa4b0d080c1f29` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:06.003938 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86565264-78eb-4ab4-919d-3df4dfdc2711 None admin] RBAC: Authorizing `identity:get_project(project_id=592a7a9f7334413584fa4b0d080c1f29)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:06.005375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86565264-78eb-4ab4-919d-3df4dfdc2711 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:06.007424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 48/93] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:05 2022] GET /identity/v3/projects/592a7a9f7334413584fa4b0d080c1f29 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:06.008744 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86565264-78eb-4ab4-919d-3df4dfdc2711 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:06.044068 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c393c-6101-4324-96c1-b55e1848badf None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:06.050920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:06.079604 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:06.117458 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86565264-78eb-4ab4-919d-3df4dfdc2711 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:06.130749 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139988b6-6221-4865-b4ec-b72bfd4b82f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:06.131417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139988b6-6221-4865-b4ec-b72bfd4b82f6 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:06.131701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139988b6-6221-4865-b4ec-b72bfd4b82f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:06.131975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139988b6-6221-4865-b4ec-b72bfd4b82f6 None admin] PATH_INFO: `/v3/projects/354eb0dc44ab47b8adc38d694c51ec13` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:06.133901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139988b6-6221-4865-b4ec-b72bfd4b82f6 None admin] RBAC: Authorizing `identity:get_project(project_id=354eb0dc44ab47b8adc38d694c51ec13)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:06.135350 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139988b6-6221-4865-b4ec-b72bfd4b82f6 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:06.142692 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 46/94] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:05 2022] GET /identity/v3/projects/354eb0dc44ab47b8adc38d694c51ec13 => generated 312 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:06.143552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139988b6-6221-4865-b4ec-b72bfd4b82f6 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:06.166135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa6ea4-2522-4f35-9732-c47e9a2170d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:06.166864 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa6ea4-2522-4f35-9732-c47e9a2170d3 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:06.167151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa6ea4-2522-4f35-9732-c47e9a2170d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:06.167426 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa6ea4-2522-4f35-9732-c47e9a2170d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:06.168223 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa6ea4-2522-4f35-9732-c47e9a2170d3 None admin] RBAC: Adding query filter params (role.id=8ce0850e15804f3d811e88cc9841e673, scope.project.id=592a7a9f7334413584fa4b0d080c1f29, user.id=5b34698a764b44aba3d550dca381a370) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:06.168531 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa6ea4-2522-4f35-9732-c47e9a2170d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:06.170093 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfa6ea4-2522-4f35-9732-c47e9a2170d3 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:06.176476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 49/95] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:46:06 2022] GET /identity/v3/role_assignments?user.id=5b34698a764b44aba3d550dca381a370&scope.project.id=592a7a9f7334413584fa4b0d080c1f29&role.id=8ce0850e15804f3d811e88cc9841e673 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:06.177373 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa6ea4-2522-4f35-9732-c47e9a2170d3 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:06.339324 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa6ea4-2522-4f35-9732-c47e9a2170d3 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:06.345345 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139988b6-6221-4865-b4ec-b72bfd4b82f6 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:06.401519 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:06.412277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c5dbac-96d8-475f-aee8-7e8f75a56ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:06.414431 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5dbac-96d8-475f-aee8-7e8f75a56ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:06.414431 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5dbac-96d8-475f-aee8-7e8f75a56ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:06.414431 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5dbac-96d8-475f-aee8-7e8f75a56ee3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:06.414431 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c5dbac-96d8-475f-aee8-7e8f75a56ee3 None admin] RBAC: Adding query filter params (role.id=111e800ddb0a411c80664cd51ce13fc6, scope.project.id=592a7a9f7334413584fa4b0d080c1f29, user.id=7e1257a546984dca904f536574103621) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:06.414839 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c5dbac-96d8-475f-aee8-7e8f75a56ee3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:06.415408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8bbb9-7ee3-4caf-a97b-16d4f1fc8aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:06.416053 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8bbb9-7ee3-4caf-a97b-16d4f1fc8aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:06.416311 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c5dbac-96d8-475f-aee8-7e8f75a56ee3 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:06.416450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8bbb9-7ee3-4caf-a97b-16d4f1fc8aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:06.416659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8bbb9-7ee3-4caf-a97b-16d4f1fc8aa2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:06.417482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8bbb9-7ee3-4caf-a97b-16d4f1fc8aa2 None admin] RBAC: Adding query filter params (role.id=68fe920c9be84b3aa1d35bdce554d96a, scope.project.id=354eb0dc44ab47b8adc38d694c51ec13, user.id=7e1257a546984dca904f536574103621) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:06.417799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8bbb9-7ee3-4caf-a97b-16d4f1fc8aa2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:06.420105 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8bbb9-7ee3-4caf-a97b-16d4f1fc8aa2 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:06.423165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 47/96] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:46:06 2022] GET /identity/v3/role_assignments?user.id=7e1257a546984dca904f536574103621&scope.project.id=592a7a9f7334413584fa4b0d080c1f29&role.id=111e800ddb0a411c80664cd51ce13fc6 => generated 259 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:06.431367 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 50/97] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:46:06 2022] GET /identity/v3/role_assignments?user.id=7e1257a546984dca904f536574103621&scope.project.id=354eb0dc44ab47b8adc38d694c51ec13&role.id=68fe920c9be84b3aa1d35bdce554d96a => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:07.370350 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a13448-b790-4eb9-9511-cf9d51d7ff8c None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:07.370844 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a13448-b790-4eb9-9511-cf9d51d7ff8c None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:07.371078 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a13448-b790-4eb9-9511-cf9d51d7ff8c None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:07.372715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 48/98] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:07.377842 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150b7dd-0ba8-4846-80b8-c783de63decb None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:07.378732 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150b7dd-0ba8-4846-80b8-c783de63decb None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:07.379206 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150b7dd-0ba8-4846-80b8-c783de63decb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:07.404151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-d150b7dd-0ba8-4846-80b8-c783de63decb None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:07.405695 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d150b7dd-0ba8-4846-80b8-c783de63decb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:07.477619 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 51/99] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:07.513373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a13448-b790-4eb9-9511-cf9d51d7ff8c None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:07.554618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:07.641762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a13448-b790-4eb9-9511-cf9d51d7ff8c None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:07.647424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609d223-e14b-43e8-a6b6-0a5e7fe5eb96 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:07.647983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609d223-e14b-43e8-a6b6-0a5e7fe5eb96 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:07.648472 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609d223-e14b-43e8-a6b6-0a5e7fe5eb96 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:07.650554 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 52/100] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:07.666629 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b375f-af90-412c-a02e-3afe7137aa70 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:07.667249 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b375f-af90-412c-a02e-3afe7137aa70 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:07.667724 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b375f-af90-412c-a02e-3afe7137aa70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:07.698350 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b8eca-5c26-4313-8c92-c9315755cd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:07.699069 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b8eca-5c26-4313-8c92-c9315755cd45 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:07.699299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b8eca-5c26-4313-8c92-c9315755cd45 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:07.699583 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b8eca-5c26-4313-8c92-c9315755cd45 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:07.703720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-6c0b375f-af90-412c-a02e-3afe7137aa70 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:07.705113 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6c0b375f-af90-412c-a02e-3afe7137aa70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:07.723398 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b8eca-5c26-4313-8c92-c9315755cd45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:07.733912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-f79b8eca-5c26-4313-8c92-c9315755cd45 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:46:07.733912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:07.733912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:07.733912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:07.733912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:07.733912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:07.733912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:07.733912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:07.733912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:07.733912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:07.733912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:07.733912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:07.733912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:07.733912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:07.733912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:07.733912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:07.734973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:07.734973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:07.734973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:07.734973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:07.734973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:07.734973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:07.734973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:07.734973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:07.734973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:07.734973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:07.734973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:07.734973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:07.734973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:07.734973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:07.734973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:07.734973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:07.735747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:07.735747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:07.735747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:07.735747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:07.735747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:07.735747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:07.735747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:07.735747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:07.735747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:07.735747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:07.735747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:46:07.735747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:07.738093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 49/101] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:07.744526 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b8eca-5c26-4313-8c92-c9315755cd45 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:07.843578 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 53/102] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 02 08:46:07.882174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b375f-af90-412c-a02e-3afe7137aa70 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:07.906456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:07.912687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b8eca-5c26-4313-8c92-c9315755cd45 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:07.968939 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f0fdd-a132-47bb-832e-7aae9adb8bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:07.969569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f0fdd-a132-47bb-832e-7aae9adb8bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:07.969859 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f0fdd-a132-47bb-832e-7aae9adb8bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:07.970167 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f0fdd-a132-47bb-832e-7aae9adb8bb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:07.970899 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f0fdd-a132-47bb-832e-7aae9adb8bb6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:07.971216 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f0fdd-a132-47bb-832e-7aae9adb8bb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:07.972615 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f0fdd-a132-47bb-832e-7aae9adb8bb6 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:07.985292 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 50/103] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:07.991055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f0fdd-a132-47bb-832e-7aae9adb8bb6 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:08.036764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:08.137444 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f0fdd-a132-47bb-832e-7aae9adb8bb6 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:08.153960 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b375f-af90-412c-a02e-3afe7137aa70 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:08.191622 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a745b-c478-4bd6-947c-c085980f6130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:08.192617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a745b-c478-4bd6-947c-c085980f6130 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:08.193094 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a745b-c478-4bd6-947c-c085980f6130 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:08.193616 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556a745b-c478-4bd6-947c-c085980f6130 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:08.195227 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a745b-c478-4bd6-947c-c085980f6130 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:08.197819 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556a745b-c478-4bd6-947c-c085980f6130 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:08.199409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5817f6b-efc9-435f-a923-0178f96714ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:08.200072 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5817f6b-efc9-435f-a923-0178f96714ed None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:08.200421 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5817f6b-efc9-435f-a923-0178f96714ed None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:08.200645 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5817f6b-efc9-435f-a923-0178f96714ed None admin] PATH_INFO: `/v3/users/ef7ba2bc6d7048b783369519072bbf60` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:08.201059 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 54/104] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:46:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:08.207009 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a745b-c478-4bd6-947c-c085980f6130 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:08.227391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5817f6b-efc9-435f-a923-0178f96714ed None admin] RBAC: Authorizing `identity:get_user(user_id=ef7ba2bc6d7048b783369519072bbf60)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:08.229010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5817f6b-efc9-435f-a923-0178f96714ed None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:08.230807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 51/105] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:07 2022] GET /identity/v3/users/ef7ba2bc6d7048b783369519072bbf60 => generated 289 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:08.236221 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5817f6b-efc9-435f-a923-0178f96714ed None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:08.290352 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:08.385658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556a745b-c478-4bd6-947c-c085980f6130 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:08.391253 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5817f6b-efc9-435f-a923-0178f96714ed None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:08.425067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:08.438450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bedbcd-f540-477e-a514-4891b7978c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:08.439081 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bedbcd-f540-477e-a514-4891b7978c87 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:08.439369 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bedbcd-f540-477e-a514-4891b7978c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:08.439654 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bedbcd-f540-477e-a514-4891b7978c87 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:08.440429 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bedbcd-f540-477e-a514-4891b7978c87 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:08.441960 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bedbcd-f540-477e-a514-4891b7978c87 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:08.469574 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b40fc-e64b-4785-8a9a-c24c4b8627b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:08.471596 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b40fc-e64b-4785-8a9a-c24c4b8627b5 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:08.471596 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b40fc-e64b-4785-8a9a-c24c4b8627b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:08.471596 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b40fc-e64b-4785-8a9a-c24c4b8627b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:08.530168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 55/106] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 2 08:46:08 2022] POST /identity/v3/projects => generated 302 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:46:08.531686 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddb6e9-0d74-4d1e-bb0d-9bfc74803015 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:08.532153 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddb6e9-0d74-4d1e-bb0d-9bfc74803015 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:08.532581 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddb6e9-0d74-4d1e-bb0d-9bfc74803015 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:08.534802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 56/107] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:08.544970 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2dc7a8-d046-45c2-9960-c12b4b80e436 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:08.545489 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2dc7a8-d046-45c2-9960-c12b4b80e436 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:08.545970 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2dc7a8-d046-45c2-9960-c12b4b80e436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:08.569212 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-3a2dc7a8-d046-45c2-9960-c12b4b80e436 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:08.570759 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3a2dc7a8-d046-45c2-9960-c12b4b80e436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:08.574267 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b40fc-e64b-4785-8a9a-c24c4b8627b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:08.576041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b40fc-e64b-4785-8a9a-c24c4b8627b5 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:08.609618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:08.704183 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 57/108] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:08.735686 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2dc7a8-d046-45c2-9960-c12b4b80e436 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:08.771034 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 52/109] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 08:46:08.779107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b40fc-e64b-4785-8a9a-c24c4b8627b5 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:08.913844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2dc7a8-d046-45c2-9960-c12b4b80e436 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:08.920849 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b40fc-e64b-4785-8a9a-c24c4b8627b5 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:08.930822 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:08.938733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:08.961157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd06d7-2528-492e-9171-d4d62b53f4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:08.961757 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd06d7-2528-492e-9171-d4d62b53f4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:08.962047 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd06d7-2528-492e-9171-d4d62b53f4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:08.962330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd06d7-2528-492e-9171-d4d62b53f4b2 None admin] PATH_INFO: `/v3/projects/592a7a9f7334413584fa4b0d080c1f29` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:08.963603 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffd06d7-2528-492e-9171-d4d62b53f4b2 None admin] RBAC: Authorizing `identity:get_project(project_id=592a7a9f7334413584fa4b0d080c1f29)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:08.965105 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffd06d7-2528-492e-9171-d4d62b53f4b2 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:08.967192 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 53/110] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:08 2022] GET /identity/v3/projects/592a7a9f7334413584fa4b0d080c1f29 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:08.973034 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd06d7-2528-492e-9171-d4d62b53f4b2 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:09.002698 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8cd336-2d6d-47ba-9ddb-22a32706dce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:09.003381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cd336-2d6d-47ba-9ddb-22a32706dce8 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:09.003676 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cd336-2d6d-47ba-9ddb-22a32706dce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:09.003961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cd336-2d6d-47ba-9ddb-22a32706dce8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:09.026859 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8cd336-2d6d-47ba-9ddb-22a32706dce8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:09.032191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-3b8cd336-2d6d-47ba-9ddb-22a32706dce8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:09.032191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:09.032191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:09.032191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:09.032191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:09.032191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:09.032191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:09.032191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:09.032191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:09.032191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:09.032191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:09.032191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:09.032191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:09.032191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:09.032191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:09.032191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:09.033102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:09.033102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:09.033102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:09.033102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:09.033102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:09.033102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:09.033102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:09.033102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:09.033102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:09.033102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:09.033102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:09.033102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:09.033102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:09.033102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:09.033102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:09.033102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:09.033867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:09.033867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:09.033867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:09.033867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:09.033867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:09.033867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:09.033867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:09.033867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:09.033867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:09.033867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:09.033867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:09.033867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:09.033867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 58/111] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:46:09.039871 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8cd336-2d6d-47ba-9ddb-22a32706dce8 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:09.082889 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd06d7-2528-492e-9171-d4d62b53f4b2 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:09.148070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf831d2-2155-4878-8605-e8486927e601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:09.149297 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf831d2-2155-4878-8605-e8486927e601 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:09.149586 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf831d2-2155-4878-8605-e8486927e601 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:09.149865 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf831d2-2155-4878-8605-e8486927e601 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:09.150691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf831d2-2155-4878-8605-e8486927e601 None admin] RBAC: Adding query filter params (role.id=3c439be3950947478195dcc08835a9b5, scope.project.id=592a7a9f7334413584fa4b0d080c1f29, user.id=ef7ba2bc6d7048b783369519072bbf60) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:09.151002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf831d2-2155-4878-8605-e8486927e601 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:09.152465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf831d2-2155-4878-8605-e8486927e601 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:09.166636 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8cd336-2d6d-47ba-9ddb-22a32706dce8 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:09.167614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 54/112] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:46:08 2022] GET /identity/v3/role_assignments?user.id=ef7ba2bc6d7048b783369519072bbf60&scope.project.id=592a7a9f7334413584fa4b0d080c1f29&role.id=3c439be3950947478195dcc08835a9b5 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:09.181606 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2859ef-7da7-420e-b0c5-61638258b9d2 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:09.181950 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2859ef-7da7-420e-b0c5-61638258b9d2 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:09.182213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2859ef-7da7-420e-b0c5-61638258b9d2 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:09.187085 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 55/113] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:09.199431 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b15bd7-d118-4824-95ce-6a74255aebc7 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:09.199759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b15bd7-d118-4824-95ce-6a74255aebc7 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:09.200003 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b15bd7-d118-4824-95ce-6a74255aebc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:09.204917 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0a605-bb2b-4a50-9f60-506a09340966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:09.205545 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0a605-bb2b-4a50-9f60-506a09340966 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:09.205833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0a605-bb2b-4a50-9f60-506a09340966 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:09.206173 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0a605-bb2b-4a50-9f60-506a09340966 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:09.206921 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0a605-bb2b-4a50-9f60-506a09340966 None admin] RBAC: Adding query filter params (name=member) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:09.207238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0a605-bb2b-4a50-9f60-506a09340966 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:09.208669 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0a605-bb2b-4a50-9f60-506a09340966 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:09.228227 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 59/114] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:09.229306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0a605-bb2b-4a50-9f60-506a09340966 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:09.238310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-b3b15bd7-d118-4824-95ce-6a74255aebc7 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:09.239520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-b3b15bd7-d118-4824-95ce-6a74255aebc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:09.288637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:09.367598 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:09.402514 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 56/115] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:09.418721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0a605-bb2b-4a50-9f60-506a09340966 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:09.433945 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b15bd7-d118-4824-95ce-6a74255aebc7 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:09.482695 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fd0be-e146-46ca-855e-4449e445d801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:09.483705 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fd0be-e146-46ca-855e-4449e445d801 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:09.484217 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fd0be-e146-46ca-855e-4449e445d801 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:09.484769 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9fd0be-e146-46ca-855e-4449e445d801 None admin] PATH_INFO: `/v3/users/7e1257a546984dca904f536574103621` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:09.486265 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9fd0be-e146-46ca-855e-4449e445d801 None admin] RBAC: Authorizing `identity:get_user(user_id=7e1257a546984dca904f536574103621)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:09.487941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9fd0be-e146-46ca-855e-4449e445d801 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:09.490076 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 60/116] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:09 2022] GET /identity/v3/users/7e1257a546984dca904f536574103621 => generated 275 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:09.508282 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fd0be-e146-46ca-855e-4449e445d801 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:09.542834 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b15bd7-d118-4824-95ce-6a74255aebc7 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:09.579082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9886bd-1a52-4d10-8810-1c15a49df33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:09.579709 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9886bd-1a52-4d10-8810-1c15a49df33a None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:09.579999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9886bd-1a52-4d10-8810-1c15a49df33a None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:09.580283 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9886bd-1a52-4d10-8810-1c15a49df33a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:09.591214 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9886bd-1a52-4d10-8810-1c15a49df33a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:09.596140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-af9886bd-1a52-4d10-8810-1c15a49df33a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:46:09.596140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:09.596140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:09.596140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:09.596140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:09.596140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:09.596140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:09.596140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:09.596140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:09.596140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:09.596140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:09.596140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:09.596140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:09.596140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:09.596140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:09.596140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:09.597028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:09.597028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:09.597028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:09.597028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:09.597028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:09.597028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:09.597028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:09.597028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:09.597028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:09.597028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:09.597028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:09.597028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:09.597028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:09.597028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:09.597028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:09.597028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:09.597802 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:09.597802 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:09.597802 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:09.597802 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:09.597802 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:09.597802 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:09.597802 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:09.597802 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:09.597802 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:09.597802 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:09.597802 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:46:09.597802 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:09.597802 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 57/117] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:46:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:46:09.603559 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9886bd-1a52-4d10-8810-1c15a49df33a None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:09.672554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:09.685276 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:09.697873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9fd0be-e146-46ca-855e-4449e445d801 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:09.739033 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9886bd-1a52-4d10-8810-1c15a49df33a None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:09.778824 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b238c-3012-4c42-9ece-8ef443244c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:09.779906 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b238c-3012-4c42-9ece-8ef443244c23 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:09.780407 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b238c-3012-4c42-9ece-8ef443244c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:09.780904 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b238c-3012-4c42-9ece-8ef443244c23 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:09.781835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b238c-3012-4c42-9ece-8ef443244c23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:09.782396 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b238c-3012-4c42-9ece-8ef443244c23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:09.784019 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b238c-3012-4c42-9ece-8ef443244c23 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:09.790778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885d680-827b-4afa-b649-c04eac4b7fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:09.794068 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885d680-827b-4afa-b649-c04eac4b7fed None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:09.794068 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885d680-827b-4afa-b649-c04eac4b7fed None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:09.794068 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885d680-827b-4afa-b649-c04eac4b7fed None admin] PATH_INFO: `/v3/projects/592a7a9f7334413584fa4b0d080c1f29` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:09.794068 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f885d680-827b-4afa-b649-c04eac4b7fed None admin] RBAC: Authorizing `identity:get_project(project_id=592a7a9f7334413584fa4b0d080c1f29)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:09.794755 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 58/118] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:09.795316 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f885d680-827b-4afa-b649-c04eac4b7fed None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:09.796582 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269ce40-9465-4e61-8e63-a3cb11246310 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:09.797261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 61/119] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:09 2022] GET /identity/v3/projects/592a7a9f7334413584fa4b0d080c1f29 => generated 298 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:09.797494 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269ce40-9465-4e61-8e63-a3cb11246310 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:09.798005 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8269ce40-9465-4e61-8e63-a3cb11246310 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:09.798392 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e413f4b-a9b0-4a9d-a901-c4a7ef2b8032 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:09.798680 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e413f4b-a9b0-4a9d-a901-c4a7ef2b8032 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:09.799099 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e413f4b-a9b0-4a9d-a901-c4a7ef2b8032 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:09.800101 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 59/120] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:09.802255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 62/121] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:09.804873 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269ce40-9465-4e61-8e63-a3cb11246310 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:09.806386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e413f4b-a9b0-4a9d-a901-c4a7ef2b8032 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:09.940726 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:09.944707 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8269ce40-9465-4e61-8e63-a3cb11246310 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:09.950125 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e413f4b-a9b0-4a9d-a901-c4a7ef2b8032 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:09.990916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc788a1-b74d-4500-9e53-cf9047831188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:09.991519 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc788a1-b74d-4500-9e53-cf9047831188 None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:09.991859 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc788a1-b74d-4500-9e53-cf9047831188 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:09.992136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc788a1-b74d-4500-9e53-cf9047831188 None admin] PATH_INFO: `/v3/projects/592a7a9f7334413584fa4b0d080c1f29/users/7e1257a546984dca904f536574103621/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:09.994070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc43bff-219f-4f12-b417-fe53706fe6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:09.994782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc43bff-219f-4f12-b417-fe53706fe6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:09.995066 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc43bff-219f-4f12-b417-fe53706fe6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:09.995339 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc43bff-219f-4f12-b417-fe53706fe6e6 None admin] PATH_INFO: `/v3/users/5b34698a764b44aba3d550dca381a370` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:09.996313 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc788a1-b74d-4500-9e53-cf9047831188 None admin] RBAC: Authorizing `identity:create_grant(project_id=592a7a9f7334413584fa4b0d080c1f29, user_id=7e1257a546984dca904f536574103621, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:09.996515 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc43bff-219f-4f12-b417-fe53706fe6e6 None admin] RBAC: Authorizing `identity:get_user(user_id=5b34698a764b44aba3d550dca381a370)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:09.998074 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc43bff-219f-4f12-b417-fe53706fe6e6 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:09.998426 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc788a1-b74d-4500-9e53-cf9047831188 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:09.999700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 60/122] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:09 2022] GET /identity/v3/users/5b34698a764b44aba3d550dca381a370 => generated 247 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:10.000807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86261344-0d88-413c-be46-7c372f1321d4 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:10.001095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86261344-0d88-413c-be46-7c372f1321d4 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:10.001373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86261344-0d88-413c-be46-7c372f1321d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:10.020521 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-86261344-0d88-413c-be46-7c372f1321d4 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:10.021698 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-86261344-0d88-413c-be46-7c372f1321d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:10.074153 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 63/123] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:46:09 2022] PUT /identity/v3/projects/592a7a9f7334413584fa4b0d080c1f29/users/7e1257a546984dca904f536574103621/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:46:10.076391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68850041-f4f3-474c-9f1f-09a967a7ca9f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:10.077234 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68850041-f4f3-474c-9f1f-09a967a7ca9f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:10.077734 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68850041-f4f3-474c-9f1f-09a967a7ca9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:10.099078 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 61/124] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:10.100076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86261344-0d88-413c-be46-7c372f1321d4 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:10.104248 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-68850041-f4f3-474c-9f1f-09a967a7ca9f None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:10.106807 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-68850041-f4f3-474c-9f1f-09a967a7ca9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:10.130446 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:10.216298 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 64/125] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:10.218457 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68850041-f4f3-474c-9f1f-09a967a7ca9f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:10.239487 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:10.245617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86261344-0d88-413c-be46-7c372f1321d4 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:10.325991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc1435e-225e-4463-9208-c34597429b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:10.327037 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1435e-225e-4463-9208-c34597429b63 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:10.327544 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1435e-225e-4463-9208-c34597429b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:10.328056 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc1435e-225e-4463-9208-c34597429b63 None admin] PATH_INFO: `/v3/projects/592a7a9f7334413584fa4b0d080c1f29` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:10.329750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc1435e-225e-4463-9208-c34597429b63 None admin] RBAC: Authorizing `identity:get_project(project_id=592a7a9f7334413584fa4b0d080c1f29)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:10.331436 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc1435e-225e-4463-9208-c34597429b63 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:10.333846 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 62/126] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:10 2022] GET /identity/v3/projects/592a7a9f7334413584fa4b0d080c1f29 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:10.335014 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc1435e-225e-4463-9208-c34597429b63 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:10.388610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68850041-f4f3-474c-9f1f-09a967a7ca9f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:10.427998 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee638a7-79e7-468a-a422-5723464d98b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:10.428646 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee638a7-79e7-468a-a422-5723464d98b4 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:10.428646 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee638a7-79e7-468a-a422-5723464d98b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:10.428785 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee638a7-79e7-468a-a422-5723464d98b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:10.442702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee638a7-79e7-468a-a422-5723464d98b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:10.450700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-8ee638a7-79e7-468a-a422-5723464d98b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:10.450700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:10.450700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:10.450700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:10.450700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:10.450700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:10.450700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:10.450700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:10.450700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:10.450700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:10.450700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:10.450700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:10.450700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:10.450700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:10.450700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:10.450700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:10.451614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:10.451614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:10.451614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:10.451614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:10.451614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:10.451614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:10.451614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:10.451614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:10.451614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:10.451614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:10.451614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:10.451614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:10.451614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:10.451614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:10.451614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:10.451614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:10.452393 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:10.452393 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:10.452393 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:10.452393 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:10.452393 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:10.452393 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:10.452393 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:10.452393 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:10.452393 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:10.452393 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:10.452393 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:10.452393 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:10.452393 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 65/127] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:10.452393 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee638a7-79e7-468a-a422-5723464d98b4 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:10.475695 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:10.483634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc1435e-225e-4463-9208-c34597429b63 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:10.533391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7bc8a-49db-430d-a279-2c50115785fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:10.534418 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7bc8a-49db-430d-a279-2c50115785fb None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:10.534909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7bc8a-49db-430d-a279-2c50115785fb None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:10.535415 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7bc8a-49db-430d-a279-2c50115785fb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:10.559370 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab7bc8a-49db-430d-a279-2c50115785fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:10.565095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-4ab7bc8a-49db-430d-a279-2c50115785fb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:10.565095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:10.565095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:10.565095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:10.565095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:10.565095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:10.565095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:10.565095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:10.565095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:10.565095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:10.565095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:10.565095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:10.565095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:10.565095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:10.565095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:10.565095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:10.565999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:10.565999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:10.565999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:10.565999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:10.565999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:10.565999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:10.565999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:10.565999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:10.565999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:10.565999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:10.565999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:10.565999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:10.565999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:10.565999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:10.565999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:10.565999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:10.566789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:10.566789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:10.566789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:10.566789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:10.566789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:10.566789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:10.566789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:10.566789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:10.566789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:10.566789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:10.566789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:10.566789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:10.577387 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 63/128] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 2 08:46:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:10.579724 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7bc8a-49db-430d-a279-2c50115785fb None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:10.636159 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:10.669878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee638a7-79e7-468a-a422-5723464d98b4 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:10.738217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab7bc8a-49db-430d-a279-2c50115785fb None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:10.750046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419a499-ac16-4c39-b3dc-502238d9037f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:10.750738 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419a499-ac16-4c39-b3dc-502238d9037f None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:10.751021 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419a499-ac16-4c39-b3dc-502238d9037f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:10.751300 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2419a499-ac16-4c39-b3dc-502238d9037f None admin] PATH_INFO: `/v3/projects/592a7a9f7334413584fa4b0d080c1f29/users/5b34698a764b44aba3d550dca381a370/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:10.754030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2419a499-ac16-4c39-b3dc-502238d9037f None admin] RBAC: Authorizing `identity:create_grant(project_id=592a7a9f7334413584fa4b0d080c1f29, user_id=5b34698a764b44aba3d550dca381a370, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:10.755800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2419a499-ac16-4c39-b3dc-502238d9037f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:10.783254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0939df0-0638-45cf-9e10-b32f20939b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:10.783986 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0939df0-0638-45cf-9e10-b32f20939b0c None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:10.784219 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0939df0-0638-45cf-9e10-b32f20939b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:10.784499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0939df0-0638-45cf-9e10-b32f20939b0c None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:10.785219 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0939df0-0638-45cf-9e10-b32f20939b0c None admin] RBAC: Adding query filter params (name=member) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:10.785531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0939df0-0638-45cf-9e10-b32f20939b0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:10.787003 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0939df0-0638-45cf-9e10-b32f20939b0c None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:10.796156 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 64/129] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:10.801221 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0939df0-0638-45cf-9e10-b32f20939b0c None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:10.879494 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 66/130] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:46:10 2022] PUT /identity/v3/projects/592a7a9f7334413584fa4b0d080c1f29/users/5b34698a764b44aba3d550dca381a370/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:46:10.880412 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419a499-ac16-4c39-b3dc-502238d9037f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:10.911921 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:10.924413 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0939df0-0638-45cf-9e10-b32f20939b0c None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:10.974513 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810790de-4b94-4766-ad2e-8c05c85e82a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:10.975327 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810790de-4b94-4766-ad2e-8c05c85e82a2 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:10.975618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810790de-4b94-4766-ad2e-8c05c85e82a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:10.975901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810790de-4b94-4766-ad2e-8c05c85e82a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:10.977213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810790de-4b94-4766-ad2e-8c05c85e82a2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:10.977529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810790de-4b94-4766-ad2e-8c05c85e82a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:10.978997 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810790de-4b94-4766-ad2e-8c05c85e82a2 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:11.007138 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 65/131] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:46:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:11.009928 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810790de-4b94-4766-ad2e-8c05c85e82a2 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:11.058359 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:11.121747 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2419a499-ac16-4c39-b3dc-502238d9037f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:11.183164 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810790de-4b94-4766-ad2e-8c05c85e82a2 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:11.196272 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c333a4-4620-4063-875a-4c8cc2ab2a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:11.196943 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c333a4-4620-4063-875a-4c8cc2ab2a15 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:11.197225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c333a4-4620-4063-875a-4c8cc2ab2a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:11.197770 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c333a4-4620-4063-875a-4c8cc2ab2a15 None admin] PATH_INFO: `/v3/users/7e1257a546984dca904f536574103621` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:11.203123 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c333a4-4620-4063-875a-4c8cc2ab2a15 None admin] RBAC: Authorizing `identity:get_user(user_id=7e1257a546984dca904f536574103621)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:11.204676 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c333a4-4620-4063-875a-4c8cc2ab2a15 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:11.210427 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 67/132] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:10 2022] GET /identity/v3/users/7e1257a546984dca904f536574103621 => generated 275 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:11.216698 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c333a4-4620-4063-875a-4c8cc2ab2a15 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:11.223019 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74bb541-4dbe-42bb-bd56-38a2e10c5bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:11.223742 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74bb541-4dbe-42bb-bd56-38a2e10c5bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:11.224030 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74bb541-4dbe-42bb-bd56-38a2e10c5bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:11.224305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74bb541-4dbe-42bb-bd56-38a2e10c5bf2 None admin] PATH_INFO: `/v3/users/7e1257a546984dca904f536574103621` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:11.225563 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74bb541-4dbe-42bb-bd56-38a2e10c5bf2 None admin] RBAC: Authorizing `identity:get_user(user_id=7e1257a546984dca904f536574103621)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:11.227208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74bb541-4dbe-42bb-bd56-38a2e10c5bf2 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:11.229193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 66/133] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:11 2022] GET /identity/v3/users/7e1257a546984dca904f536574103621 => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:11.234116 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74bb541-4dbe-42bb-bd56-38a2e10c5bf2 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:11.254748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:11.340287 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74bb541-4dbe-42bb-bd56-38a2e10c5bf2 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:11.375780 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c333a4-4620-4063-875a-4c8cc2ab2a15 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:11.376362 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9c629-4aef-41c6-8ccf-152419857485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:11.377019 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9c629-4aef-41c6-8ccf-152419857485 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:11.377307 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9c629-4aef-41c6-8ccf-152419857485 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:11.377583 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9c629-4aef-41c6-8ccf-152419857485 None admin] PATH_INFO: `/v3/projects/592a7a9f7334413584fa4b0d080c1f29` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:11.382992 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9c629-4aef-41c6-8ccf-152419857485 None admin] RBAC: Authorizing `identity:get_project(project_id=592a7a9f7334413584fa4b0d080c1f29)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:11.384453 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9c629-4aef-41c6-8ccf-152419857485 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:11.387010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 67/134] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:11 2022] GET /identity/v3/projects/592a7a9f7334413584fa4b0d080c1f29 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:11.397304 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9c629-4aef-41c6-8ccf-152419857485 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:11.440148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1620a-366b-408e-aec5-f8ecc80eaf68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:11.440795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1620a-366b-408e-aec5-f8ecc80eaf68 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:11.441083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1620a-366b-408e-aec5-f8ecc80eaf68 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:11.441363 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1620a-366b-408e-aec5-f8ecc80eaf68 None admin] PATH_INFO: `/v3/projects/354eb0dc44ab47b8adc38d694c51ec13` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:11.442571 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1620a-366b-408e-aec5-f8ecc80eaf68 None admin] RBAC: Authorizing `identity:get_project(project_id=354eb0dc44ab47b8adc38d694c51ec13)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:11.445541 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1620a-366b-408e-aec5-f8ecc80eaf68 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:11.447329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 68/135] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:11 2022] GET /identity/v3/projects/354eb0dc44ab47b8adc38d694c51ec13 => generated 312 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:11.454794 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1620a-366b-408e-aec5-f8ecc80eaf68 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:11.480652 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:11.551474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:11.566758 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9c629-4aef-41c6-8ccf-152419857485 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:11.609701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75740bfa-8426-41c5-8831-29d7bc351103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:11.610335 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75740bfa-8426-41c5-8831-29d7bc351103 None admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:11.610641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75740bfa-8426-41c5-8831-29d7bc351103 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:11.611039 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75740bfa-8426-41c5-8831-29d7bc351103 None admin] PATH_INFO: `/v3/projects/592a7a9f7334413584fa4b0d080c1f29/users/7e1257a546984dca904f536574103621/roles/111e800ddb0a411c80664cd51ce13fc6` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:11.613414 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75740bfa-8426-41c5-8831-29d7bc351103 None admin] RBAC: Authorizing `identity:create_grant(project_id=592a7a9f7334413584fa4b0d080c1f29, user_id=7e1257a546984dca904f536574103621, role_id=111e800ddb0a411c80664cd51ce13fc6)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:11.615228 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75740bfa-8426-41c5-8831-29d7bc351103 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:11.658996 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1620a-366b-408e-aec5-f8ecc80eaf68 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:11.709045 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 68/136] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:46:11 2022] PUT /identity/v3/projects/592a7a9f7334413584fa4b0d080c1f29/users/7e1257a546984dca904f536574103621/roles/111e800ddb0a411c80664cd51ce13fc6 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:46:11.718353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ff59c-17e9-4a3b-af6c-2d52c7a16e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:11.719161 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ff59c-17e9-4a3b-af6c-2d52c7a16e7b None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:11.719456 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ff59c-17e9-4a3b-af6c-2d52c7a16e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:11.719740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ff59c-17e9-4a3b-af6c-2d52c7a16e7b None admin] PATH_INFO: `/v3/projects/354eb0dc44ab47b8adc38d694c51ec13/users/7e1257a546984dca904f536574103621/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:11.721915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ff59c-17e9-4a3b-af6c-2d52c7a16e7b None admin] RBAC: Authorizing `identity:create_grant(project_id=354eb0dc44ab47b8adc38d694c51ec13, user_id=7e1257a546984dca904f536574103621, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:11.723545 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ff59c-17e9-4a3b-af6c-2d52c7a16e7b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:11.824255 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab54951-9b0d-4dc0-b428-b3cabb1092ea None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:11.824584 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab54951-9b0d-4dc0-b428-b3cabb1092ea None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:11.824829 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab54951-9b0d-4dc0-b428-b3cabb1092ea None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:11.826312 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 69/137] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:46:11.829306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 69/138] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:46:11 2022] PUT /identity/v3/projects/354eb0dc44ab47b8adc38d694c51ec13/users/7e1257a546984dca904f536574103621/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:46:11.833091 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc884d11-f3c1-4f1b-80b2-d7bcfc2748ce None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:11.833679 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc884d11-f3c1-4f1b-80b2-d7bcfc2748ce None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:11.834195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc884d11-f3c1-4f1b-80b2-d7bcfc2748ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:11.855537 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-dc884d11-f3c1-4f1b-80b2-d7bcfc2748ce None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:11.857175 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-dc884d11-f3c1-4f1b-80b2-d7bcfc2748ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:11.907969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60836fe0-b0ca-4802-9a9c-b1a217fd758b None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:11.908594 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60836fe0-b0ca-4802-9a9c-b1a217fd758b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:11.909107 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60836fe0-b0ca-4802-9a9c-b1a217fd758b None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:11.910846 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 70/139] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:11.931063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1cc58-a265-43bf-bd1b-11ab92871ce1 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:11.931374 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1cc58-a265-43bf-bd1b-11ab92871ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:11.931634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c1cc58-a265-43bf-bd1b-11ab92871ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:11.966772 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-08c1cc58-a265-43bf-bd1b-11ab92871ce1 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:11.968029 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-08c1cc58-a265-43bf-bd1b-11ab92871ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:11.980431 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 70/140] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:12.047003 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc884d11-f3c1-4f1b-80b2-d7bcfc2748ce None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:12.063188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:12.069970 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 71/141] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:46:12.111678 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1cc58-a265-43bf-bd1b-11ab92871ce1 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:12.186721 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:12.224098 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc884d11-f3c1-4f1b-80b2-d7bcfc2748ce None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:12.266156 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c1cc58-a265-43bf-bd1b-11ab92871ce1 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:12.291193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471cae51-6322-47bc-ac93-9192be10eeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:12.291872 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cae51-6322-47bc-ac93-9192be10eeda None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:12.292157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cae51-6322-47bc-ac93-9192be10eeda None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:12.292437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cae51-6322-47bc-ac93-9192be10eeda None admin] PATH_INFO: `/v3/domains/default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:12.293594 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471cae51-6322-47bc-ac93-9192be10eeda None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:12.295114 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471cae51-6322-47bc-ac93-9192be10eeda None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:12.302459 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 71/142] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:46:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:12.307833 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471cae51-6322-47bc-ac93-9192be10eeda None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:12.332817 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93828954-7b34-4822-b397-696596e5a867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:12.333490 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93828954-7b34-4822-b397-696596e5a867 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:12.333780 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93828954-7b34-4822-b397-696596e5a867 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:12.334089 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93828954-7b34-4822-b397-696596e5a867 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:12.357579 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93828954-7b34-4822-b397-696596e5a867 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:12.363526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-93828954-7b34-4822-b397-696596e5a867 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:46:12.363526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:12.363526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:12.363526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:12.363526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:12.363526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:12.363526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:12.363526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:12.363526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:12.363526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:12.363526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:12.363526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:12.363526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:12.363526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:12.363526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:12.363526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:12.364397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:12.364397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:12.364397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:12.364397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:12.364397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:12.364397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:12.364397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:12.364397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:12.364397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:12.364397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:12.364397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:12.364397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:12.364397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:12.364397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:12.364397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:12.364397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:12.365174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:12.365174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:12.365174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:12.365174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:12.365174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:12.365174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:12.365174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:12.365174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:12.365174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:12.365174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:12.365174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:46:12.365174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:12.365174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 72/143] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:46:12.371640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93828954-7b34-4822-b397-696596e5a867 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:12.501535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:12.510296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471cae51-6322-47bc-ac93-9192be10eeda None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:12.575008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886445c-c222-4b80-8d23-de6295d26e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:12.576513 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886445c-c222-4b80-8d23-de6295d26e11 None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:12.577712 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886445c-c222-4b80-8d23-de6295d26e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:12.578720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2886445c-c222-4b80-8d23-de6295d26e11 None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:12.579744 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886445c-c222-4b80-8d23-de6295d26e11 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:12.581736 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93828954-7b34-4822-b397-696596e5a867 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:12.584793 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2886445c-c222-4b80-8d23-de6295d26e11 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:12.631560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241ad37-7455-4838-a637-e2da91028e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:12.632600 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241ad37-7455-4838-a637-e2da91028e5b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:12.633059 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241ad37-7455-4838-a637-e2da91028e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:12.633563 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241ad37-7455-4838-a637-e2da91028e5b None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:12.634498 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3241ad37-7455-4838-a637-e2da91028e5b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:12.634967 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3241ad37-7455-4838-a637-e2da91028e5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:12.636621 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3241ad37-7455-4838-a637-e2da91028e5b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:12.647355 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 73/144] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:12.652979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241ad37-7455-4838-a637-e2da91028e5b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:12.730818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:12.735400 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 72/145] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 2 08:46:12 2022] POST /identity/v3/users => generated 283 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:46:12.795387 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241ad37-7455-4838-a637-e2da91028e5b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:12.818293 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:12.835624 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862addd-db9f-4d90-af2b-6e40c8c315d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:12.837169 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862addd-db9f-4d90-af2b-6e40c8c315d7 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:12.837623 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862addd-db9f-4d90-af2b-6e40c8c315d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:12.838085 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862addd-db9f-4d90-af2b-6e40c8c315d7 None admin] PATH_INFO: `/v3/users/ef7ba2bc6d7048b783369519072bbf60` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:12.839561 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862addd-db9f-4d90-af2b-6e40c8c315d7 None admin] RBAC: Authorizing `identity:get_user(user_id=ef7ba2bc6d7048b783369519072bbf60)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:12.841312 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862addd-db9f-4d90-af2b-6e40c8c315d7 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:12.843463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 74/146] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:12 2022] GET /identity/v3/users/ef7ba2bc6d7048b783369519072bbf60 => generated 289 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:12.850155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886445c-c222-4b80-8d23-de6295d26e11 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:12.982207 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2886445c-c222-4b80-8d23-de6295d26e11 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:13.025662 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26585626-30f2-43cc-a3af-cdd1a88701e6 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:13.026068 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26585626-30f2-43cc-a3af-cdd1a88701e6 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:13.026355 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26585626-30f2-43cc-a3af-cdd1a88701e6 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:13.027976 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 75/147] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:13.032751 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573130f7-7cc8-4488-8abc-b9daeff5a1d9 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:13.033040 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573130f7-7cc8-4488-8abc-b9daeff5a1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:13.033318 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573130f7-7cc8-4488-8abc-b9daeff5a1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:13.050599 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f1a283-7dae-4525-9987-79a1a0a6e95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:13.051184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1a283-7dae-4525-9987-79a1a0a6e95b None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:13.051470 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1a283-7dae-4525-9987-79a1a0a6e95b None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:13.051805 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1a283-7dae-4525-9987-79a1a0a6e95b None admin] PATH_INFO: `/v3/projects/592a7a9f7334413584fa4b0d080c1f29` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:13.052993 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f1a283-7dae-4525-9987-79a1a0a6e95b None admin] RBAC: Authorizing `identity:get_project(project_id=592a7a9f7334413584fa4b0d080c1f29)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:13.055215 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f1a283-7dae-4525-9987-79a1a0a6e95b None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:13.057087 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 73/148] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:12 2022] GET /identity/v3/projects/592a7a9f7334413584fa4b0d080c1f29 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:13.060988 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-573130f7-7cc8-4488-8abc-b9daeff5a1d9 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:13.062411 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-573130f7-7cc8-4488-8abc-b9daeff5a1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:13.062830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f1a283-7dae-4525-9987-79a1a0a6e95b None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:13.146947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 76/149] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:46:13.162372 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:13.191401 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573130f7-7cc8-4488-8abc-b9daeff5a1d9 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:13.215411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f1a283-7dae-4525-9987-79a1a0a6e95b None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:13.277157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e792af-c0e9-4594-928d-9097afe54abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:13.277911 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e792af-c0e9-4594-928d-9097afe54abe None admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:13.278236 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e792af-c0e9-4594-928d-9097afe54abe None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:13.278517 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e792af-c0e9-4594-928d-9097afe54abe None admin] PATH_INFO: `/v3/projects/592a7a9f7334413584fa4b0d080c1f29/users/ef7ba2bc6d7048b783369519072bbf60/roles/3c439be3950947478195dcc08835a9b5` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:13.280637 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e792af-c0e9-4594-928d-9097afe54abe None admin] RBAC: Authorizing `identity:create_grant(project_id=592a7a9f7334413584fa4b0d080c1f29, user_id=ef7ba2bc6d7048b783369519072bbf60, role_id=3c439be3950947478195dcc08835a9b5)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:13.282324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e792af-c0e9-4594-928d-9097afe54abe None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:13.347680 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573130f7-7cc8-4488-8abc-b9daeff5a1d9 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:13.369575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 74/150] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:46:13 2022] PUT /identity/v3/projects/592a7a9f7334413584fa4b0d080c1f29/users/ef7ba2bc6d7048b783369519072bbf60/roles/3c439be3950947478195dcc08835a9b5 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:46:13.386295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:13.404475 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824cbe0-1dee-4cd8-bb18-f944eca173cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:13.405166 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824cbe0-1dee-4cd8-bb18-f944eca173cd None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:13.405462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824cbe0-1dee-4cd8-bb18-f944eca173cd None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:13.405732 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b824cbe0-1dee-4cd8-bb18-f944eca173cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:13.420518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b824cbe0-1dee-4cd8-bb18-f944eca173cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:13.428627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-b824cbe0-1dee-4cd8-bb18-f944eca173cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:13.428627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:13.428627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:13.428627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:13.428627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:13.428627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:13.428627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:13.428627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:13.428627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:13.428627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:13.428627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:13.428627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:13.428627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:13.428627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:13.428627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:13.428627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:13.429613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:13.429613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:13.429613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:13.429613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:13.429613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:13.429613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:13.429613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:13.429613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:13.429613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:13.429613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:13.429613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:13.429613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:13.429613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:13.429613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:13.429613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:13.429613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:13.431025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:13.431025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:13.431025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:13.431025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:13.431025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:13.431025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:13.431025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:13.431025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:13.431025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:13.431025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:13.431025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:13.431025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:13.431025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 77/151] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:46:13.436636 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e792af-c0e9-4594-928d-9097afe54abe None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:13.563213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e792af-c0e9-4594-928d-9097afe54abe None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:13.598594 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:13.609555 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71106516-008b-4d4c-b9d1-444ce530a97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:13.617994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71106516-008b-4d4c-b9d1-444ce530a97a None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:13.617994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71106516-008b-4d4c-b9d1-444ce530a97a None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:13.617994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71106516-008b-4d4c-b9d1-444ce530a97a None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:13.617994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71106516-008b-4d4c-b9d1-444ce530a97a None admin] RBAC: Adding query filter params (name=member) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:13.617994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71106516-008b-4d4c-b9d1-444ce530a97a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:13.617994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71106516-008b-4d4c-b9d1-444ce530a97a None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:13.625901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 75/152] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:13.632150 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824cbe0-1dee-4cd8-bb18-f944eca173cd None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:13.747789 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b824cbe0-1dee-4cd8-bb18-f944eca173cd None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:13.784928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24789968-eecd-46b2-bb08-e62d803b8575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:13.785587 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24789968-eecd-46b2-bb08-e62d803b8575 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:13.788429 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24789968-eecd-46b2-bb08-e62d803b8575 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:13.788429 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24789968-eecd-46b2-bb08-e62d803b8575 None admin] PATH_INFO: `/v3/users/7e1257a546984dca904f536574103621` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:13.788429 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24789968-eecd-46b2-bb08-e62d803b8575 None admin] RBAC: Authorizing `identity:get_user(user_id=7e1257a546984dca904f536574103621)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:13.788950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24789968-eecd-46b2-bb08-e62d803b8575 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:13.791317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 78/153] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:13 2022] GET /identity/v3/users/7e1257a546984dca904f536574103621 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:13.797160 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71106516-008b-4d4c-b9d1-444ce530a97a None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:13.917677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71106516-008b-4d4c-b9d1-444ce530a97a None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:13.955858 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89efb69-dceb-4b59-a314-4d04720b9fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:13.956503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89efb69-dceb-4b59-a314-4d04720b9fcd None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:13.956797 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89efb69-dceb-4b59-a314-4d04720b9fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:13.957080 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89efb69-dceb-4b59-a314-4d04720b9fcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:14.020668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:14.037066 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89efb69-dceb-4b59-a314-4d04720b9fcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:14.038416 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89efb69-dceb-4b59-a314-4d04720b9fcd None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:14.158834 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 76/154] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:46:14.166425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24789968-eecd-46b2-bb08-e62d803b8575 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:14.241301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:14.306362 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24789968-eecd-46b2-bb08-e62d803b8575 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:14.338690 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d04351-88bd-40fa-aa65-1337774d662e None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:14.339430 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d04351-88bd-40fa-aa65-1337774d662e None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:14.340036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d04351-88bd-40fa-aa65-1337774d662e None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:14.341616 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 77/155] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:46:14.349525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c22a800-56a1-4ff1-bd3f-2b30e4227fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:14.350280 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c22a800-56a1-4ff1-bd3f-2b30e4227fba None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:14.350567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c22a800-56a1-4ff1-bd3f-2b30e4227fba None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:14.350846 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c22a800-56a1-4ff1-bd3f-2b30e4227fba None admin] PATH_INFO: `/v3/projects/592a7a9f7334413584fa4b0d080c1f29` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:14.352920 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c22a800-56a1-4ff1-bd3f-2b30e4227fba None admin] RBAC: Authorizing `identity:get_project(project_id=592a7a9f7334413584fa4b0d080c1f29)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:14.353241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b41d4f-ff41-41e6-8023-5af467f7a3e1 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:14.353785 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b41d4f-ff41-41e6-8023-5af467f7a3e1 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:14.354315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b41d4f-ff41-41e6-8023-5af467f7a3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:14.354429 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c22a800-56a1-4ff1-bd3f-2b30e4227fba None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:14.356115 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 79/156] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:14 2022] GET /identity/v3/projects/592a7a9f7334413584fa4b0d080c1f29 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:14.362159 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c22a800-56a1-4ff1-bd3f-2b30e4227fba None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:14.384815 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-e6b41d4f-ff41-41e6-8023-5af467f7a3e1 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:14.386405 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e6b41d4f-ff41-41e6-8023-5af467f7a3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:14.476784 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 78/157] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:46:14.516571 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b41d4f-ff41-41e6-8023-5af467f7a3e1 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:14.556907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:14.576140 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c22a800-56a1-4ff1-bd3f-2b30e4227fba None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:14.614262 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:14.665756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae24a8-88c4-496d-a3fd-e0805e48383a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:14.666422 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae24a8-88c4-496d-a3fd-e0805e48383a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:14.666684 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae24a8-88c4-496d-a3fd-e0805e48383a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:14.666966 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaae24a8-88c4-496d-a3fd-e0805e48383a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:14.667776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae24a8-88c4-496d-a3fd-e0805e48383a None admin] RBAC: Adding query filter params (role.id=68fe920c9be84b3aa1d35bdce554d96a, scope.project.id=592a7a9f7334413584fa4b0d080c1f29, user.id=7e1257a546984dca904f536574103621) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:14.668089 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae24a8-88c4-496d-a3fd-e0805e48383a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:14.669559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaae24a8-88c4-496d-a3fd-e0805e48383a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:14.682492 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b41d4f-ff41-41e6-8023-5af467f7a3e1 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:14.693065 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 80/158] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:46:14 2022] GET /identity/v3/role_assignments?user.id=7e1257a546984dca904f536574103621&scope.project.id=592a7a9f7334413584fa4b0d080c1f29&role.id=68fe920c9be84b3aa1d35bdce554d96a => generated 608 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:14.739025 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf5320d-21bb-42c6-8432-f6f3584ce1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:14.739699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf5320d-21bb-42c6-8432-f6f3584ce1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:14.739987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf5320d-21bb-42c6-8432-f6f3584ce1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:14.740267 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf5320d-21bb-42c6-8432-f6f3584ce1d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:14.761546 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf5320d-21bb-42c6-8432-f6f3584ce1d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:14.766661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-9cf5320d-21bb-42c6-8432-f6f3584ce1d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:46:14.766661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:14.766661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:14.766661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:14.766661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:14.766661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:14.766661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:14.766661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:14.766661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:14.766661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:14.766661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:14.766661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:14.766661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:14.766661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:14.766661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:14.766661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:14.767536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:14.767536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:14.767536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:14.767536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:14.767536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:14.767536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:14.767536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:14.767536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:14.767536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:14.767536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:14.767536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:14.767536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:14.767536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:14.767536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:14.767536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:14.767536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:14.768781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:14.768781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:14.768781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:14.768781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:14.768781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:14.768781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:14.768781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:14.768781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:14.768781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:14.768781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:14.768781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:46:14.768781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:14.768781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 79/159] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:46:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:46:14.776396 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae24a8-88c4-496d-a3fd-e0805e48383a None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:14.891198 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaae24a8-88c4-496d-a3fd-e0805e48383a None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:14.940444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3ef9c-87d5-4ce5-b23f-1115b5f13138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:14.940444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3ef9c-87d5-4ce5-b23f-1115b5f13138 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:14.940444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3ef9c-87d5-4ce5-b23f-1115b5f13138 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:14.940444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3ef9c-87d5-4ce5-b23f-1115b5f13138 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:14.940773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3ef9c-87d5-4ce5-b23f-1115b5f13138 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:14.940901 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3ef9c-87d5-4ce5-b23f-1115b5f13138 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:14.942271 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3ef9c-87d5-4ce5-b23f-1115b5f13138 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:14.951988 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 81/160] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:14.957747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf5320d-21bb-42c6-8432-f6f3584ce1d1 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:14.973738 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4234e3-5f69-4ea7-ba5e-eb68c7249207 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:14.974100 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4234e3-5f69-4ea7-ba5e-eb68c7249207 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:14.974329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4234e3-5f69-4ea7-ba5e-eb68c7249207 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:14.975703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 82/161] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:14.983407 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731677a-bafb-437f-85f2-3f1aff8409c9 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:14.983712 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731677a-bafb-437f-85f2-3f1aff8409c9 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:14.983962 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731677a-bafb-437f-85f2-3f1aff8409c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:15.019012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-4731677a-bafb-437f-85f2-3f1aff8409c9 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:15.020173 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4731677a-bafb-437f-85f2-3f1aff8409c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:15.084122 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:15.138225 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:15.154616 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 83/162] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:15.155714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22ecf6-65be-4d9e-a527-2d571c1e6233 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:15.156005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22ecf6-65be-4d9e-a527-2d571c1e6233 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:15.156310 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f22ecf6-65be-4d9e-a527-2d571c1e6233 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:15.157564 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 84/163] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:15.171776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93c5b2-ab91-4400-91cf-852ffb9d426b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:15.172063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93c5b2-ab91-4400-91cf-852ffb9d426b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:15.172343 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93c5b2-ab91-4400-91cf-852ffb9d426b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:15.179079 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf5320d-21bb-42c6-8432-f6f3584ce1d1 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:15.206498 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-fd93c5b2-ab91-4400-91cf-852ffb9d426b None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:15.207723 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-fd93c5b2-ab91-4400-91cf-852ffb9d426b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:15.256895 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71171e2c-2c6f-4174-b044-a296f8f4f990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:15.257556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71171e2c-2c6f-4174-b044-a296f8f4f990 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:15.257989 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71171e2c-2c6f-4174-b044-a296f8f4f990 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:15.258268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71171e2c-2c6f-4174-b044-a296f8f4f990 None admin] PATH_INFO: `/v3/users/5b34698a764b44aba3d550dca381a370` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:15.259659 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71171e2c-2c6f-4174-b044-a296f8f4f990 None admin] RBAC: Authorizing `identity:get_user(user_id=5b34698a764b44aba3d550dca381a370)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:15.261116 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71171e2c-2c6f-4174-b044-a296f8f4f990 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:15.262777 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 80/164] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:14 2022] GET /identity/v3/users/5b34698a764b44aba3d550dca381a370 => generated 247 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:15.263553 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71171e2c-2c6f-4174-b044-a296f8f4f990 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:15.385820 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71171e2c-2c6f-4174-b044-a296f8f4f990 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:15.395523 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 85/165] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:46:15.396188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93c5b2-ab91-4400-91cf-852ffb9d426b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:15.427064 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d713a4-702b-400a-8c46-babfcaef1747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:15.427976 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d713a4-702b-400a-8c46-babfcaef1747 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:15.428410 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d713a4-702b-400a-8c46-babfcaef1747 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:15.428835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d713a4-702b-400a-8c46-babfcaef1747 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:15.441217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d713a4-702b-400a-8c46-babfcaef1747 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:15.448144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-43d713a4-702b-400a-8c46-babfcaef1747 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:15.448144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:15.448144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:15.448144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:15.448144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:15.448144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:15.448144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:15.448144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:15.448144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:15.448144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:15.448144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:15.448144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:15.448144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:15.448144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:15.448144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:15.448144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:15.449195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:15.449195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:15.449195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:15.449195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:15.449195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:15.449195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:15.449195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:15.449195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:15.449195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:15.449195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:15.449195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:15.449195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:15.449195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:15.449195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:15.449195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:15.449195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:15.450155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:15.450155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:15.450155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:15.450155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:15.450155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:15.450155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:15.450155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:15.450155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:15.450155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:15.450155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:15.450155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:15.450155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:15.452385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 81/166] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 2 08:46:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:15.453289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d713a4-702b-400a-8c46-babfcaef1747 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:15.466698 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:15.540210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:15.548615 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93c5b2-ab91-4400-91cf-852ffb9d426b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:15.565332 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d713a4-702b-400a-8c46-babfcaef1747 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:15.588496 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09fd7f-4ccd-4911-9762-b4b734cc64ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:15.589490 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09fd7f-4ccd-4911-9762-b4b734cc64ad None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:15.589959 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09fd7f-4ccd-4911-9762-b4b734cc64ad None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:15.590385 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb09fd7f-4ccd-4911-9762-b4b734cc64ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:15.602119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98edbb-d4c1-48a9-8de7-43159ec68c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:15.602996 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98edbb-d4c1-48a9-8de7-43159ec68c98 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:15.603420 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98edbb-d4c1-48a9-8de7-43159ec68c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:15.603849 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98edbb-d4c1-48a9-8de7-43159ec68c98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:15.616182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98edbb-d4c1-48a9-8de7-43159ec68c98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:15.622496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-7f98edbb-d4c1-48a9-8de7-43159ec68c98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:15.622496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:15.622496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:15.622496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:15.622496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:15.622496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:15.622496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:15.622496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:15.622496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:15.622496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:15.622496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:15.622496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:15.622496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:15.622496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:15.622496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:15.622496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:15.623573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:15.623573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:15.623573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:15.623573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:15.623573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:15.623573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:15.623573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:15.623573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:15.623573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:15.623573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:15.623573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:15.623573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:15.623573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:15.623573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:15.623573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:15.623573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:15.624558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:15.624558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:15.624558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:15.624558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:15.624558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:15.624558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:15.624558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:15.624558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:15.624558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:15.624558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:15.624558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:15.624558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:15.626660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 82/167] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:15.627580 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98edbb-d4c1-48a9-8de7-43159ec68c98 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:15.666639 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09fd7f-4ccd-4911-9762-b4b734cc64ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:15.668231 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb09fd7f-4ccd-4911-9762-b4b734cc64ad None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:15.696656 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:15.731920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98edbb-d4c1-48a9-8de7-43159ec68c98 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:15.769835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b488c-3943-46d7-9726-d5cc841b1513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:15.770718 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b488c-3943-46d7-9726-d5cc841b1513 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:15.771140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b488c-3943-46d7-9726-d5cc841b1513 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:15.771583 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b488c-3943-46d7-9726-d5cc841b1513 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:15.772218 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:15.772585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b488c-3943-46d7-9726-d5cc841b1513 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:15.773028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b488c-3943-46d7-9726-d5cc841b1513 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:15.774855 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b488c-3943-46d7-9726-d5cc841b1513 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:15.775808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 86/168] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:46:15.776631 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09fd7f-4ccd-4911-9762-b4b734cc64ad None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:15.784569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 83/169] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:46:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:15.785444 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b488c-3943-46d7-9726-d5cc841b1513 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:15.881678 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb09fd7f-4ccd-4911-9762-b4b734cc64ad None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:15.896150 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b488c-3943-46d7-9726-d5cc841b1513 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:15.918918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f87ed-f61e-4623-bfd5-2e7d3db47a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:15.919783 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f87ed-f61e-4623-bfd5-2e7d3db47a64 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:15.920213 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f87ed-f61e-4623-bfd5-2e7d3db47a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:15.920635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f87ed-f61e-4623-bfd5-2e7d3db47a64 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:15.921499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f87ed-f61e-4623-bfd5-2e7d3db47a64 None admin] RBAC: Adding query filter params (name=member) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:15.922005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f87ed-f61e-4623-bfd5-2e7d3db47a64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:15.923542 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f87ed-f61e-4623-bfd5-2e7d3db47a64 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:15.928688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:15.933130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 87/170] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:15.934196 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f87ed-f61e-4623-bfd5-2e7d3db47a64 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:15.935857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec693550-c57e-4db6-971c-552b5b659c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:15.944292 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec693550-c57e-4db6-971c-552b5b659c96 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:15.944292 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec693550-c57e-4db6-971c-552b5b659c96 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:15.944292 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec693550-c57e-4db6-971c-552b5b659c96 None admin] PATH_INFO: `/v3/projects/592a7a9f7334413584fa4b0d080c1f29` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:15.944292 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec693550-c57e-4db6-971c-552b5b659c96 None admin] RBAC: Authorizing `identity:get_project(project_id=592a7a9f7334413584fa4b0d080c1f29)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:15.944292 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec693550-c57e-4db6-971c-552b5b659c96 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:15.944292 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 84/171] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:15 2022] GET /identity/v3/projects/592a7a9f7334413584fa4b0d080c1f29 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:15.948508 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec693550-c57e-4db6-971c-552b5b659c96 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:16.014343 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:16.040442 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f87ed-f61e-4623-bfd5-2e7d3db47a64 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:16.086563 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01059ef3-65b5-497f-ab97-b72c2eebf09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:16.087224 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01059ef3-65b5-497f-ab97-b72c2eebf09c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:16.087502 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01059ef3-65b5-497f-ab97-b72c2eebf09c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:16.087772 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01059ef3-65b5-497f-ab97-b72c2eebf09c None admin] PATH_INFO: `/v3/users/7e1257a546984dca904f536574103621` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:16.089362 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01059ef3-65b5-497f-ab97-b72c2eebf09c None admin] RBAC: Authorizing `identity:get_user(user_id=7e1257a546984dca904f536574103621)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:16.091206 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01059ef3-65b5-497f-ab97-b72c2eebf09c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:16.093221 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 88/172] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:15 2022] GET /identity/v3/users/7e1257a546984dca904f536574103621 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:16.094230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01059ef3-65b5-497f-ab97-b72c2eebf09c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:16.144454 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec693550-c57e-4db6-971c-552b5b659c96 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:16.204957 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1a2a6-ddc9-43f1-bfba-708dc13af5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:16.205992 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1a2a6-ddc9-43f1-bfba-708dc13af5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:16.206484 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1a2a6-ddc9-43f1-bfba-708dc13af5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:16.206979 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1a2a6-ddc9-43f1-bfba-708dc13af5e6 None admin] PATH_INFO: `/v3/users/7e1257a546984dca904f536574103621` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:16.210109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1a2a6-ddc9-43f1-bfba-708dc13af5e6 None admin] RBAC: Authorizing `identity:get_user(user_id=7e1257a546984dca904f536574103621)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:16.211954 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1a2a6-ddc9-43f1-bfba-708dc13af5e6 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:16.213808 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 85/173] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:15 2022] GET /identity/v3/users/7e1257a546984dca904f536574103621 => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:16.214826 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1a2a6-ddc9-43f1-bfba-708dc13af5e6 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:16.325097 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:16.378970 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01059ef3-65b5-497f-ab97-b72c2eebf09c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:16.400237 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1a2a6-ddc9-43f1-bfba-708dc13af5e6 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:16.431541 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:16.438803 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c3545-91e0-41db-b7fc-47278a67c9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:16.439426 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c3545-91e0-41db-b7fc-47278a67c9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:16.445768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c3545-91e0-41db-b7fc-47278a67c9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:16.445768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1c3545-91e0-41db-b7fc-47278a67c9b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:16.445768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c3545-91e0-41db-b7fc-47278a67c9b2 None admin] RBAC: Adding query filter params (role.id=8ce0850e15804f3d811e88cc9841e673, scope.project.id=592a7a9f7334413584fa4b0d080c1f29, user.id=5b34698a764b44aba3d550dca381a370) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:16.445768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c3545-91e0-41db-b7fc-47278a67c9b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:16.445768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1c3545-91e0-41db-b7fc-47278a67c9b2 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:16.454124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 89/174] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:46:16 2022] GET /identity/v3/role_assignments?user.id=5b34698a764b44aba3d550dca381a370&scope.project.id=592a7a9f7334413584fa4b0d080c1f29&role.id=8ce0850e15804f3d811e88cc9841e673 => generated 608 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:16.455243 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c3545-91e0-41db-b7fc-47278a67c9b2 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:16.491358 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf97bb-3816-4d7f-977f-552c0ce6c64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:16.492939 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf97bb-3816-4d7f-977f-552c0ce6c64f None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:16.493430 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf97bb-3816-4d7f-977f-552c0ce6c64f None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:16.493925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf97bb-3816-4d7f-977f-552c0ce6c64f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:16.625090 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1c3545-91e0-41db-b7fc-47278a67c9b2 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:16.690669 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81ccdb-6b78-4dc2-bc1a-203441e805f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:16.691288 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81ccdb-6b78-4dc2-bc1a-203441e805f2 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:16.691578 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81ccdb-6b78-4dc2-bc1a-203441e805f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:16.691857 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81ccdb-6b78-4dc2-bc1a-203441e805f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:16.757527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf97bb-3816-4d7f-977f-552c0ce6c64f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:16.760672 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf97bb-3816-4d7f-977f-552c0ce6c64f None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:16.798237 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:16.810191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81ccdb-6b78-4dc2-bc1a-203441e805f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:16.811674 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81ccdb-6b78-4dc2-bc1a-203441e805f2 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:16.884220 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:16.960012 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 86/175] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 745 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:46:16.961436 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a101d-e686-41f7-92c7-cd53e9c3819e None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:16.961724 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a101d-e686-41f7-92c7-cd53e9c3819e None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:16.962036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a101d-e686-41f7-92c7-cd53e9c3819e None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:16.963302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 87/176] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:16.968356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a101d-e686-41f7-92c7-cd53e9c3819e None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:17.022917 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 90/177] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:46:17.024563 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82f7d0-812e-403f-b033-da0daccf56b0 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:17.024848 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82f7d0-812e-403f-b033-da0daccf56b0 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:17.025126 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82f7d0-812e-403f-b033-da0daccf56b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:17.061721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-da82f7d0-812e-403f-b033-da0daccf56b0 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:17.063267 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-da82f7d0-812e-403f-b033-da0daccf56b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:17.095534 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a101d-e686-41f7-92c7-cd53e9c3819e None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:17.149229 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f0bfc-e05a-4d81-b532-bb744689209b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:17.149892 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f0bfc-e05a-4d81-b532-bb744689209b None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:17.150180 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f0bfc-e05a-4d81-b532-bb744689209b None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:17.150455 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304f0bfc-e05a-4d81-b532-bb744689209b None admin] PATH_INFO: `/v3/projects/592a7a9f7334413584fa4b0d080c1f29` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:17.154133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f0bfc-e05a-4d81-b532-bb744689209b None admin] RBAC: Authorizing `identity:get_project(project_id=592a7a9f7334413584fa4b0d080c1f29)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:17.155539 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304f0bfc-e05a-4d81-b532-bb744689209b None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:17.157243 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 88/178] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:16 2022] GET /identity/v3/projects/592a7a9f7334413584fa4b0d080c1f29 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:17.158033 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f0bfc-e05a-4d81-b532-bb744689209b None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:17.200641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 91/179] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:17.201324 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82f7d0-812e-403f-b033-da0daccf56b0 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:17.225226 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:17.250934 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:17.349097 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304f0bfc-e05a-4d81-b532-bb744689209b None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:17.388800 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c7af9-174c-461e-a057-57594b1eb291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:17.389444 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c7af9-174c-461e-a057-57594b1eb291 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:17.389721 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c7af9-174c-461e-a057-57594b1eb291 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:17.390021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080c7af9-174c-461e-a057-57594b1eb291 None admin] PATH_INFO: `/v3/projects/354eb0dc44ab47b8adc38d694c51ec13` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:17.391571 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080c7af9-174c-461e-a057-57594b1eb291 None admin] RBAC: Authorizing `identity:get_project(project_id=354eb0dc44ab47b8adc38d694c51ec13)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:17.393021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080c7af9-174c-461e-a057-57594b1eb291 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:17.395237 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 89/180] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:17 2022] GET /identity/v3/projects/354eb0dc44ab47b8adc38d694c51ec13 => generated 312 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:17.395821 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c7af9-174c-461e-a057-57594b1eb291 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:17.409844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82f7d0-812e-403f-b033-da0daccf56b0 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:17.448508 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505df571-fa53-4e46-9458-fe4e5bcb551c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:17.449239 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505df571-fa53-4e46-9458-fe4e5bcb551c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:17.449593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505df571-fa53-4e46-9458-fe4e5bcb551c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:17.449968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505df571-fa53-4e46-9458-fe4e5bcb551c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:17.450840 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505df571-fa53-4e46-9458-fe4e5bcb551c None admin] RBAC: Adding query filter params (role.id=111e800ddb0a411c80664cd51ce13fc6, scope.project.id=592a7a9f7334413584fa4b0d080c1f29, user.id=7e1257a546984dca904f536574103621) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:17.451242 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505df571-fa53-4e46-9458-fe4e5bcb551c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:17.452784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505df571-fa53-4e46-9458-fe4e5bcb551c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:17.459177 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 92/181] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:46:17 2022] GET /identity/v3/role_assignments?user.id=7e1257a546984dca904f536574103621&scope.project.id=592a7a9f7334413584fa4b0d080c1f29&role.id=111e800ddb0a411c80664cd51ce13fc6 => generated 608 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:17.460637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755090a-e537-45e6-b48b-0d4f72eab8b0 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:17.461539 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755090a-e537-45e6-b48b-0d4f72eab8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:17.461830 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755090a-e537-45e6-b48b-0d4f72eab8b0 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:17.463450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 93/182] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:17.464585 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755090a-e537-45e6-b48b-0d4f72eab8b0 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:17.525990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080c7af9-174c-461e-a057-57594b1eb291 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:17.528190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:17.563375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:17.567289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1034902f-1201-45c6-94b3-8b25987b850f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:17.568217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034902f-1201-45c6-94b3-8b25987b850f None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:17.568733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034902f-1201-45c6-94b3-8b25987b850f None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:17.569207 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1034902f-1201-45c6-94b3-8b25987b850f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:17.578950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755090a-e537-45e6-b48b-0d4f72eab8b0 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:17.584272 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1034902f-1201-45c6-94b3-8b25987b850f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:17.590627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-1034902f-1201-45c6-94b3-8b25987b850f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:46:17.590627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:17.590627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:17.590627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:17.590627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:17.590627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:17.590627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:17.590627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:17.590627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:17.590627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:17.590627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:17.590627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:17.590627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:17.590627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:17.590627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:17.590627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:17.591529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:17.591529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:17.591529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:17.591529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:17.591529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:17.591529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:17.591529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:17.591529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:17.591529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:17.591529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:17.591529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:17.591529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:17.591529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:17.591529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:17.591529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:17.591529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:17.592324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:17.592324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:17.592324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:17.592324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:17.592324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:17.592324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:17.592324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:17.592324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:17.592324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:17.592324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:17.592324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:46:17.592324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:17.594632 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 90/183] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:46:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:17.596216 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659493b-bb9b-4c46-b2f4-b32bceab89be None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:17.596746 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659493b-bb9b-4c46-b2f4-b32bceab89be None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:17.597231 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2659493b-bb9b-4c46-b2f4-b32bceab89be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:17.619326 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-2659493b-bb9b-4c46-b2f4-b32bceab89be None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:17.621179 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-2659493b-bb9b-4c46-b2f4-b32bceab89be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:17.622926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34448e2b-8281-4fc4-ad4a-6629a76b8811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:17.623576 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34448e2b-8281-4fc4-ad4a-6629a76b8811 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:17.623859 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34448e2b-8281-4fc4-ad4a-6629a76b8811 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:17.624138 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34448e2b-8281-4fc4-ad4a-6629a76b8811 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:17.624964 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34448e2b-8281-4fc4-ad4a-6629a76b8811 None admin] RBAC: Adding query filter params (role.id=68fe920c9be84b3aa1d35bdce554d96a, scope.project.id=354eb0dc44ab47b8adc38d694c51ec13, user.id=7e1257a546984dca904f536574103621) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:17.625282 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34448e2b-8281-4fc4-ad4a-6629a76b8811 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:17.627097 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34448e2b-8281-4fc4-ad4a-6629a76b8811 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:17.633924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 94/184] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:46:17 2022] GET /identity/v3/role_assignments?user.id=7e1257a546984dca904f536574103621&scope.project.id=354eb0dc44ab47b8adc38d694c51ec13&role.id=68fe920c9be84b3aa1d35bdce554d96a => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:17.638592 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34448e2b-8281-4fc4-ad4a-6629a76b8811 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:17.709710 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 91/185] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:17.712120 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdff56-3c0c-46e4-9f46-c87eaa33de2e None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:17.712691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdff56-3c0c-46e4-9f46-c87eaa33de2e None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:17.713222 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfdff56-3c0c-46e4-9f46-c87eaa33de2e None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:17.715207 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 92/186] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:17.728666 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31056e00-a9d4-48b9-9a0a-3f99562f7bb9 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:17.729337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31056e00-a9d4-48b9-9a0a-3f99562f7bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:17.729839 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31056e00-a9d4-48b9-9a0a-3f99562f7bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:17.763000 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-31056e00-a9d4-48b9-9a0a-3f99562f7bb9 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:17.764302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-31056e00-a9d4-48b9-9a0a-3f99562f7bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:17.833751 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34448e2b-8281-4fc4-ad4a-6629a76b8811 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:17.871262 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 93/187] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:17.872035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31056e00-a9d4-48b9-9a0a-3f99562f7bb9 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:17.884454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f737e9-a361-45ae-a075-f982344647ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:17.885087 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f737e9-a361-45ae-a075-f982344647ec None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:17.885358 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f737e9-a361-45ae-a075-f982344647ec None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:17.885637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f737e9-a361-45ae-a075-f982344647ec None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:17.890092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f737e9-a361-45ae-a075-f982344647ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:17.890092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f737e9-a361-45ae-a075-f982344647ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:17.890092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f737e9-a361-45ae-a075-f982344647ec None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:17.901115 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:17.902981 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 95/188] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:17.905479 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aaa8eb-2c97-405f-99e1-15ebb25c6d6f None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:17.905839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aaa8eb-2c97-405f-99e1-15ebb25c6d6f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:17.906756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aaa8eb-2c97-405f-99e1-15ebb25c6d6f None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:17.907538 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 96/189] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:17.908612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aaa8eb-2c97-405f-99e1-15ebb25c6d6f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:17.927575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:17.986583 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31056e00-a9d4-48b9-9a0a-3f99562f7bb9 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:18.015838 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aaa8eb-2c97-405f-99e1-15ebb25c6d6f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:18.020100 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec6d4b-fa48-4525-929a-2aaf4a33c5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:18.020730 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec6d4b-fa48-4525-929a-2aaf4a33c5be None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:18.021008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec6d4b-fa48-4525-929a-2aaf4a33c5be None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:18.021277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dec6d4b-fa48-4525-929a-2aaf4a33c5be None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:18.031094 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dec6d4b-fa48-4525-929a-2aaf4a33c5be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:18.036077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-4dec6d4b-fa48-4525-929a-2aaf4a33c5be None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:46:18.036077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:18.036077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:18.036077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:18.036077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:18.036077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:18.036077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:18.036077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:18.036077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:18.036077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:18.036077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:18.036077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:18.036077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:18.036077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:18.036077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:18.036077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:18.037112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:18.037112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:18.037112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:18.037112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:18.037112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:18.037112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:18.037112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:18.037112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:18.037112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:18.037112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:18.037112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:18.037112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:18.037112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:18.037112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:18.037112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:18.037112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:18.037957 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:18.037957 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:18.037957 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:18.037957 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:18.037957 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:18.037957 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:18.037957 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:18.037957 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:18.037957 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:18.037957 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:18.037957 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:46:18.037957 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:18.037957 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 94/190] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:18.038640 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec6d4b-fa48-4525-929a-2aaf4a33c5be None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:18.052562 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967e940-1957-4337-8e9a-1e5b8c1cbb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:18.053207 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967e940-1957-4337-8e9a-1e5b8c1cbb5b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:18.053493 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967e940-1957-4337-8e9a-1e5b8c1cbb5b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:18.053778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967e940-1957-4337-8e9a-1e5b8c1cbb5b None admin] PATH_INFO: `/v3/users/430270d2a8664baab8b8a0415d1dbb24` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:18.067711 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967e940-1957-4337-8e9a-1e5b8c1cbb5b None admin] RBAC: Authorizing `identity:get_user(user_id=430270d2a8664baab8b8a0415d1dbb24)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:18.069143 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967e940-1957-4337-8e9a-1e5b8c1cbb5b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:18.070842 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 97/191] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:17 2022] GET /identity/v3/users/430270d2a8664baab8b8a0415d1dbb24 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:18.075239 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b371c6-01b8-471d-9a5e-a4c5ad6ea350 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:18.075602 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b371c6-01b8-471d-9a5e-a4c5ad6ea350 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:18.075888 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b371c6-01b8-471d-9a5e-a4c5ad6ea350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:18.092896 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-d0b371c6-01b8-471d-9a5e-a4c5ad6ea350 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:18.094111 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d0b371c6-01b8-471d-9a5e-a4c5ad6ea350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:18.119442 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:18.136875 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dec6d4b-fa48-4525-929a-2aaf4a33c5be None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:18.160252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 98/192] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:18.161084 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b371c6-01b8-471d-9a5e-a4c5ad6ea350 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:18.163243 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:18.175480 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c33268-97a4-4329-b07f-98230e98ba21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:18.176130 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c33268-97a4-4329-b07f-98230e98ba21 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:18.176589 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c33268-97a4-4329-b07f-98230e98ba21 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:18.176865 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c33268-97a4-4329-b07f-98230e98ba21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:18.178690 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c33268-97a4-4329-b07f-98230e98ba21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:18.181193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c33268-97a4-4329-b07f-98230e98ba21 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:18.183020 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 95/193] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:46:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:18.183917 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c33268-97a4-4329-b07f-98230e98ba21 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:18.266327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b371c6-01b8-471d-9a5e-a4c5ad6ea350 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:18.283734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c33268-97a4-4329-b07f-98230e98ba21 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:18.300730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ec5da-2b33-4cd7-98f5-08aeea8cfafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:18.301350 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ec5da-2b33-4cd7-98f5-08aeea8cfafb None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:18.301630 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ec5da-2b33-4cd7-98f5-08aeea8cfafb None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:18.301921 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ec5da-2b33-4cd7-98f5-08aeea8cfafb None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:18.302636 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ec5da-2b33-4cd7-98f5-08aeea8cfafb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:18.302938 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ec5da-2b33-4cd7-98f5-08aeea8cfafb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:18.304319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ec5da-2b33-4cd7-98f5-08aeea8cfafb None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:18.312375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 99/194] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:18.313060 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ec5da-2b33-4cd7-98f5-08aeea8cfafb None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:18.321097 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d9a33-11b5-4df7-bcf3-18f4837161aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:18.321722 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d9a33-11b5-4df7-bcf3-18f4837161aa None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:18.322051 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d9a33-11b5-4df7-bcf3-18f4837161aa None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:18.322334 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d9a33-11b5-4df7-bcf3-18f4837161aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:18.341716 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:18.375156 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:18.391672 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d9a33-11b5-4df7-bcf3-18f4837161aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:18.393064 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d9a33-11b5-4df7-bcf3-18f4837161aa None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:18.414121 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ec5da-2b33-4cd7-98f5-08aeea8cfafb None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:18.447229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d271ad1-326b-4632-83ba-54bc10bc9c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:18.448126 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d271ad1-326b-4632-83ba-54bc10bc9c44 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:18.448407 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d271ad1-326b-4632-83ba-54bc10bc9c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:18.448686 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d271ad1-326b-4632-83ba-54bc10bc9c44 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:18.458596 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d271ad1-326b-4632-83ba-54bc10bc9c44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:18.463900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-1d271ad1-326b-4632-83ba-54bc10bc9c44 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:18.463900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:18.463900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:18.463900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:18.463900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:18.463900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:18.463900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:18.463900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:18.463900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:18.463900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:18.463900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:18.463900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:18.463900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:18.463900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:18.463900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:18.463900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:18.464969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:18.464969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:18.464969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:18.464969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:18.464969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:18.464969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:18.464969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:18.464969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:18.464969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:18.464969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:18.464969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:18.464969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:18.464969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:18.464969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:18.464969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:18.464969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:18.465797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:18.465797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:18.465797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:18.465797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:18.465797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:18.465797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:18.465797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:18.465797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:18.465797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:18.465797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:18.465797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:18.465797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:18.465797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 100/195] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 2 08:46:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:18.465797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d271ad1-326b-4632-83ba-54bc10bc9c44 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:18.492843 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 96/196] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:46:18.493741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d9a33-11b5-4df7-bcf3-18f4837161aa None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:18.541564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:18.568622 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d271ad1-326b-4632-83ba-54bc10bc9c44 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:18.591368 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:18.591937 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d9a33-11b5-4df7-bcf3-18f4837161aa None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:18.602148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b9419-cd2c-4efc-8975-79507519884b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:18.602760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b9419-cd2c-4efc-8975-79507519884b None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:18.603039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b9419-cd2c-4efc-8975-79507519884b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:18.603313 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657b9419-cd2c-4efc-8975-79507519884b None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:18.603989 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b9419-cd2c-4efc-8975-79507519884b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:18.605427 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657b9419-cd2c-4efc-8975-79507519884b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:18.625849 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1353e6-7c70-4812-a813-cdca52af75e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:18.630420 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1353e6-7c70-4812-a813-cdca52af75e2 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:18.630420 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1353e6-7c70-4812-a813-cdca52af75e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:18.630420 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1353e6-7c70-4812-a813-cdca52af75e2 None admin] PATH_INFO: `/v3/users/ef7ba2bc6d7048b783369519072bbf60` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:18.630420 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1353e6-7c70-4812-a813-cdca52af75e2 None admin] RBAC: Authorizing `identity:get_user(user_id=ef7ba2bc6d7048b783369519072bbf60)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:18.630420 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1353e6-7c70-4812-a813-cdca52af75e2 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:18.631203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 97/197] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:18 2022] GET /identity/v3/users/ef7ba2bc6d7048b783369519072bbf60 => generated 289 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:18.632032 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1353e6-7c70-4812-a813-cdca52af75e2 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:18.705140 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 101/198] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 2 08:46:18 2022] POST /identity/v3/users => generated 297 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:46:18.716251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b9419-cd2c-4efc-8975-79507519884b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:18.734703 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1353e6-7c70-4812-a813-cdca52af75e2 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:18.756272 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:18.779288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d6db5-4f47-4b90-bd2f-fc9d6e1f38a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:18.780672 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d6db5-4f47-4b90-bd2f-fc9d6e1f38a0 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:18.781076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d6db5-4f47-4b90-bd2f-fc9d6e1f38a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:18.781353 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112d6db5-4f47-4b90-bd2f-fc9d6e1f38a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:18.782129 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112d6db5-4f47-4b90-bd2f-fc9d6e1f38a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:18.782435 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112d6db5-4f47-4b90-bd2f-fc9d6e1f38a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:18.783845 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112d6db5-4f47-4b90-bd2f-fc9d6e1f38a0 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:18.795665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 98/199] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:46:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:18.796451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d6db5-4f47-4b90-bd2f-fc9d6e1f38a0 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:18.847185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657b9419-cd2c-4efc-8975-79507519884b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:18.890553 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbbaab-5613-412b-b36f-7eac5dfba32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:18.891217 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbbaab-5613-412b-b36f-7eac5dfba32d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:18.891501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbbaab-5613-412b-b36f-7eac5dfba32d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:18.892098 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbbaab-5613-412b-b36f-7eac5dfba32d None admin] PATH_INFO: `/v3/projects/f8bf83f1361247688208d846845c9d82` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:18.893299 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbbaab-5613-412b-b36f-7eac5dfba32d None admin] RBAC: Authorizing `identity:get_project(project_id=f8bf83f1361247688208d846845c9d82)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:18.894830 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbbaab-5613-412b-b36f-7eac5dfba32d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:18.896517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 102/200] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:18 2022] GET /identity/v3/projects/f8bf83f1361247688208d846845c9d82 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:18.897273 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbbaab-5613-412b-b36f-7eac5dfba32d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:18.915330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112d6db5-4f47-4b90-bd2f-fc9d6e1f38a0 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:18.959643 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:18.964231 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a900f4-ef26-46b4-ab0a-636ba28d31f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:18.964635 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a900f4-ef26-46b4-ab0a-636ba28d31f7 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:18.964921 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a900f4-ef26-46b4-ab0a-636ba28d31f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:18.965204 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a900f4-ef26-46b4-ab0a-636ba28d31f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:19.027431 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbbaab-5613-412b-b36f-7eac5dfba32d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:19.032354 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:19.055026 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a900f4-ef26-46b4-ab0a-636ba28d31f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:19.056621 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a900f4-ef26-46b4-ab0a-636ba28d31f7 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:19.071816 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d4688-b382-46f8-8e16-8e8d28494b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:19.072749 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d4688-b382-46f8-8e16-8e8d28494b4c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:19.073035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d4688-b382-46f8-8e16-8e8d28494b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:19.073308 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d4688-b382-46f8-8e16-8e8d28494b4c None admin] PATH_INFO: `/v3/users/430270d2a8664baab8b8a0415d1dbb24` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:19.077850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d4688-b382-46f8-8e16-8e8d28494b4c None admin] RBAC: Authorizing `identity:get_user(user_id=430270d2a8664baab8b8a0415d1dbb24)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:19.077850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122d4688-b382-46f8-8e16-8e8d28494b4c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:19.077850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 103/201] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:18 2022] GET /identity/v3/users/430270d2a8664baab8b8a0415d1dbb24 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:19.079456 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d4688-b382-46f8-8e16-8e8d28494b4c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:19.181041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 99/202] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:46:19.181917 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a900f4-ef26-46b4-ab0a-636ba28d31f7 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:19.224708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d4688-b382-46f8-8e16-8e8d28494b4c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:19.247115 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:19.269918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cead2c-1c79-4994-b9c9-61150ec35994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:19.270849 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cead2c-1c79-4994-b9c9-61150ec35994 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:19.271278 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cead2c-1c79-4994-b9c9-61150ec35994 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:19.271727 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cead2c-1c79-4994-b9c9-61150ec35994 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:19.272673 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cead2c-1c79-4994-b9c9-61150ec35994 None admin] RBAC: Adding query filter params (role.id=8ce0850e15804f3d811e88cc9841e673, scope.project.id=f8bf83f1361247688208d846845c9d82, user.id=430270d2a8664baab8b8a0415d1dbb24) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:19.273131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cead2c-1c79-4994-b9c9-61150ec35994 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:19.274772 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:19.276161 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cead2c-1c79-4994-b9c9-61150ec35994 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:19.282220 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 104/203] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:46:19 2022] GET /identity/v3/role_assignments?user.id=430270d2a8664baab8b8a0415d1dbb24&scope.project.id=f8bf83f1361247688208d846845c9d82&role.id=8ce0850e15804f3d811e88cc9841e673 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:19.283204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cead2c-1c79-4994-b9c9-61150ec35994 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:19.298653 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a900f4-ef26-46b4-ab0a-636ba28d31f7 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:19.374736 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bda609-5e10-4c5e-bd1c-e26bf1004afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:19.377426 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bda609-5e10-4c5e-bd1c-e26bf1004afc None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:19.378686 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bda609-5e10-4c5e-bd1c-e26bf1004afc None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:19.380511 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bda609-5e10-4c5e-bd1c-e26bf1004afc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:19.449769 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cead2c-1c79-4994-b9c9-61150ec35994 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:19.498267 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bda609-5e10-4c5e-bd1c-e26bf1004afc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:19.499465 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf22a5b-bd26-427e-8777-949c128911ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:19.499933 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bda609-5e10-4c5e-bd1c-e26bf1004afc None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:19.500447 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf22a5b-bd26-427e-8777-949c128911ff None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:19.500934 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf22a5b-bd26-427e-8777-949c128911ff None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:19.501353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf22a5b-bd26-427e-8777-949c128911ff None admin] PATH_INFO: `/v3/projects/592a7a9f7334413584fa4b0d080c1f29` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:19.502944 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf22a5b-bd26-427e-8777-949c128911ff None admin] RBAC: Authorizing `identity:get_project(project_id=592a7a9f7334413584fa4b0d080c1f29)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:19.505439 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf22a5b-bd26-427e-8777-949c128911ff None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:19.507661 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 105/204] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:19 2022] GET /identity/v3/projects/592a7a9f7334413584fa4b0d080c1f29 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:19.517917 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf22a5b-bd26-427e-8777-949c128911ff None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:19.622277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:19.624563 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:19.645977 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 100/205] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:46:19.656943 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bda609-5e10-4c5e-bd1c-e26bf1004afc None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:19.725376 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf22a5b-bd26-427e-8777-949c128911ff None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:19.761260 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b06758-e1aa-4d82-9aed-b747efef6505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:19.762242 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b06758-e1aa-4d82-9aed-b747efef6505 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:19.762750 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b06758-e1aa-4d82-9aed-b747efef6505 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:19.763224 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b06758-e1aa-4d82-9aed-b747efef6505 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:19.764218 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b06758-e1aa-4d82-9aed-b747efef6505 None admin] RBAC: Adding query filter params (role.id=3c439be3950947478195dcc08835a9b5, scope.project.id=592a7a9f7334413584fa4b0d080c1f29, user.id=ef7ba2bc6d7048b783369519072bbf60) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:19.764731 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b06758-e1aa-4d82-9aed-b747efef6505 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:19.766397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b06758-e1aa-4d82-9aed-b747efef6505 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:19.772959 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 106/206] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:46:19 2022] GET /identity/v3/role_assignments?user.id=ef7ba2bc6d7048b783369519072bbf60&scope.project.id=592a7a9f7334413584fa4b0d080c1f29&role.id=3c439be3950947478195dcc08835a9b5 => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:19.863753 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bda609-5e10-4c5e-bd1c-e26bf1004afc None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:19.927844 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e9426-3497-47d5-8a17-24c24e15d714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:19.928873 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e9426-3497-47d5-8a17-24c24e15d714 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:19.929358 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e9426-3497-47d5-8a17-24c24e15d714 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:19.931355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e9426-3497-47d5-8a17-24c24e15d714 None admin] PATH_INFO: `/v3/projects/f8bf83f1361247688208d846845c9d82` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:19.932875 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e9426-3497-47d5-8a17-24c24e15d714 None admin] RBAC: Authorizing `identity:get_project(project_id=f8bf83f1361247688208d846845c9d82)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:19.934617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e9426-3497-47d5-8a17-24c24e15d714 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:19.936657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 101/207] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:19 2022] GET /identity/v3/projects/f8bf83f1361247688208d846845c9d82 => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:19.942840 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b06758-e1aa-4d82-9aed-b747efef6505 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:20.064387 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b06758-e1aa-4d82-9aed-b747efef6505 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:20.078089 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:20.099714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89430bbc-10ca-43d6-a02b-1c32cdfb4330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:20.100344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89430bbc-10ca-43d6-a02b-1c32cdfb4330 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:20.100625 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89430bbc-10ca-43d6-a02b-1c32cdfb4330 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:20.107421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89430bbc-10ca-43d6-a02b-1c32cdfb4330 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:20.107421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89430bbc-10ca-43d6-a02b-1c32cdfb4330 None admin] RBAC: Adding query filter params (role.id=111e800ddb0a411c80664cd51ce13fc6, scope.project.id=f8bf83f1361247688208d846845c9d82, user.id=430270d2a8664baab8b8a0415d1dbb24) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:20.107421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89430bbc-10ca-43d6-a02b-1c32cdfb4330 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:20.107421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89430bbc-10ca-43d6-a02b-1c32cdfb4330 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:20.110677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 107/208] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:46:19 2022] GET /identity/v3/role_assignments?user.id=430270d2a8664baab8b8a0415d1dbb24&scope.project.id=f8bf83f1361247688208d846845c9d82&role.id=111e800ddb0a411c80664cd51ce13fc6 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:21.022483 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8402e8-31e2-4732-8837-d06449c84f63 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:21.022992 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8402e8-31e2-4732-8837-d06449c84f63 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:21.023099 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8402e8-31e2-4732-8837-d06449c84f63 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:21.025049 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 102/209] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:21.030552 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb4742-5e20-4afe-af8b-673af47d6838 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:21.030962 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb4742-5e20-4afe-af8b-673af47d6838 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:21.031128 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abb4742-5e20-4afe-af8b-673af47d6838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:21.049551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5abb4742-5e20-4afe-af8b-673af47d6838 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:21.050800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5abb4742-5e20-4afe-af8b-673af47d6838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:21.121264 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 108/210] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:21.144791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8402e8-31e2-4732-8837-d06449c84f63 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:21.173504 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1de53e-a9cf-4dea-add4-60b8f2c05424 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:21.173811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1de53e-a9cf-4dea-add4-60b8f2c05424 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:21.174234 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1de53e-a9cf-4dea-add4-60b8f2c05424 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:21.175760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 109/211] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:46:21.182616 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78de0d5-65c5-492a-9577-61394da0ccbc None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:21.182901 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78de0d5-65c5-492a-9577-61394da0ccbc None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:21.183200 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78de0d5-65c5-492a-9577-61394da0ccbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:21.202192 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-c78de0d5-65c5-492a-9577-61394da0ccbc None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:21.203149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c78de0d5-65c5-492a-9577-61394da0ccbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:21.205146 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:21.271485 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8402e8-31e2-4732-8837-d06449c84f63 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:21.297928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 110/212] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:46:21.316414 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78de0d5-65c5-492a-9577-61394da0ccbc None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:21.321954 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fe87e-9717-4e36-8e7e-d5e93a3e13a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:21.322992 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fe87e-9717-4e36-8e7e-d5e93a3e13a9 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:21.323368 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fe87e-9717-4e36-8e7e-d5e93a3e13a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:21.323714 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56fe87e-9717-4e36-8e7e-d5e93a3e13a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:21.334220 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56fe87e-9717-4e36-8e7e-d5e93a3e13a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:21.339266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-b56fe87e-9717-4e36-8e7e-d5e93a3e13a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:21.339266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:21.339266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:21.339266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:21.339266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:21.339266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:21.339266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:21.339266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:21.339266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:21.339266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:21.339266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:21.339266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:21.339266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:21.339266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:21.339266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:21.339266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:21.340186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:21.340186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:21.340186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:21.340186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:21.340186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:21.340186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:21.340186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:21.340186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:21.340186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:21.340186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:21.340186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:21.340186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:21.340186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:21.340186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:21.340186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:21.340186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:21.340987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:21.340987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:21.340987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:21.340987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:21.340987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:21.340987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:21.340987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:21.340987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:21.340987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:21.340987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:21.340987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:21.340987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:21.340987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 103/213] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:21.346262 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fe87e-9717-4e36-8e7e-d5e93a3e13a9 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:21.358847 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:21.432438 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78de0d5-65c5-492a-9577-61394da0ccbc None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:21.473746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31eb5f1-0a6b-4f1c-8104-9e1b4ce23429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:21.474527 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31eb5f1-0a6b-4f1c-8104-9e1b4ce23429 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:21.474944 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31eb5f1-0a6b-4f1c-8104-9e1b4ce23429 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:21.475321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31eb5f1-0a6b-4f1c-8104-9e1b4ce23429 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:21.477182 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31eb5f1-0a6b-4f1c-8104-9e1b4ce23429 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:21.478468 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56fe87e-9717-4e36-8e7e-d5e93a3e13a9 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:21.478868 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31eb5f1-0a6b-4f1c-8104-9e1b4ce23429 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:21.480579 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 111/214] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:46:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:21.486465 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31eb5f1-0a6b-4f1c-8104-9e1b4ce23429 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:21.494372 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:21.529395 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe82f33-576e-419a-9e9c-185ed7f5f897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:21.530047 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe82f33-576e-419a-9e9c-185ed7f5f897 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:21.530331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe82f33-576e-419a-9e9c-185ed7f5f897 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:21.530606 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe82f33-576e-419a-9e9c-185ed7f5f897 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:21.531325 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe82f33-576e-419a-9e9c-185ed7f5f897 None admin] RBAC: Adding query filter params (name=member) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:21.531630 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe82f33-576e-419a-9e9c-185ed7f5f897 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:21.533010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe82f33-576e-419a-9e9c-185ed7f5f897 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:21.545912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 104/215] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:21.545912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a280bdc-caff-4087-8ffe-161ce5952a66 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:21.545912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a280bdc-caff-4087-8ffe-161ce5952a66 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:21.545912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a280bdc-caff-4087-8ffe-161ce5952a66 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:21.545912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 105/216] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:21.550030 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a280bdc-caff-4087-8ffe-161ce5952a66 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:21.598120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31eb5f1-0a6b-4f1c-8104-9e1b4ce23429 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:21.601160 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:21.634938 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9759e-c583-4847-af21-4239e1d2344b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:21.635564 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9759e-c583-4847-af21-4239e1d2344b None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:21.635862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9759e-c583-4847-af21-4239e1d2344b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:21.636130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d9759e-c583-4847-af21-4239e1d2344b None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:21.636818 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9759e-c583-4847-af21-4239e1d2344b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:21.638300 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d9759e-c583-4847-af21-4239e1d2344b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:21.670523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a280bdc-caff-4087-8ffe-161ce5952a66 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:21.706305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db9872-a124-4b02-bd46-2bed512cb305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:21.706946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db9872-a124-4b02-bd46-2bed512cb305 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:21.707228 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db9872-a124-4b02-bd46-2bed512cb305 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:21.707502 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db9872-a124-4b02-bd46-2bed512cb305 None admin] PATH_INFO: `/v3/users/1394e6b314554594bf617f454f136113` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:21.720911 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db9872-a124-4b02-bd46-2bed512cb305 None admin] RBAC: Authorizing `identity:get_user(user_id=1394e6b314554594bf617f454f136113)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:21.722397 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db9872-a124-4b02-bd46-2bed512cb305 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:21.724200 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 106/217] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:21 2022] GET /identity/v3/users/1394e6b314554594bf617f454f136113 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:21.725289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78c6ac-3edb-4211-9cf6-32e15bab012a None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:21.725573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78c6ac-3edb-4211-9cf6-32e15bab012a None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:21.725844 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf78c6ac-3edb-4211-9cf6-32e15bab012a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:21.742586 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 112/218] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 2 08:46:21 2022] POST /identity/v3/users => generated 297 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:46:21.743883 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9759e-c583-4847-af21-4239e1d2344b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:21.744269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-cf78c6ac-3edb-4211-9cf6-32e15bab012a None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:21.745595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-cf78c6ac-3edb-4211-9cf6-32e15bab012a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:21.777931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:21.853916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 107/219] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:21.876948 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d9759e-c583-4847-af21-4239e1d2344b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:21.880209 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78c6ac-3edb-4211-9cf6-32e15bab012a None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:21.927121 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828d80d-9c8e-4946-aca0-59c943e62486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:21.928142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828d80d-9c8e-4946-aca0-59c943e62486 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:21.928632 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828d80d-9c8e-4946-aca0-59c943e62486 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:21.929099 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828d80d-9c8e-4946-aca0-59c943e62486 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:21.968484 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:22.002523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf78c6ac-3edb-4211-9cf6-32e15bab012a None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:22.012834 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3828d80d-9c8e-4946-aca0-59c943e62486 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:22.014344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3828d80d-9c8e-4946-aca0-59c943e62486 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:22.051302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa372e3-c73a-4ab9-9713-7ba6f1263bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:22.051996 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa372e3-c73a-4ab9-9713-7ba6f1263bdb None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:22.052346 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa372e3-c73a-4ab9-9713-7ba6f1263bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:22.052642 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa372e3-c73a-4ab9-9713-7ba6f1263bdb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:22.066555 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa372e3-c73a-4ab9-9713-7ba6f1263bdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:22.071782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-6aa372e3-c73a-4ab9-9713-7ba6f1263bdb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:46:22.071782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:22.071782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:22.071782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:22.071782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:22.071782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:22.071782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:22.071782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:22.071782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:22.071782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:22.071782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:22.071782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:22.071782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:22.071782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:22.071782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:22.071782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:22.072990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:22.072990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:22.072990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:22.072990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:22.072990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:22.072990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:22.072990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:22.072990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:22.072990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:22.072990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:22.072990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:22.072990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:22.072990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:22.072990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:22.072990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:22.072990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:22.074040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:22.074040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:22.074040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:22.074040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:22.074040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:22.074040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:22.074040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:22.074040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:22.074040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:22.074040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:22.074040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:46:22.074040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:22.074040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 108/220] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:46:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:22.082771 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa372e3-c73a-4ab9-9713-7ba6f1263bdb None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:22.111751 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:22.162161 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 113/221] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:46:22.168371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828d80d-9c8e-4946-aca0-59c943e62486 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:22.217224 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa372e3-c73a-4ab9-9713-7ba6f1263bdb None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:22.250523 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:22.266951 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a007ffa-5f34-4f21-98dd-5929712d66d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:22.266951 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a007ffa-5f34-4f21-98dd-5929712d66d0 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:22.266951 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a007ffa-5f34-4f21-98dd-5929712d66d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:22.266951 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a007ffa-5f34-4f21-98dd-5929712d66d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:22.267598 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a007ffa-5f34-4f21-98dd-5929712d66d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:22.267913 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a007ffa-5f34-4f21-98dd-5929712d66d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:22.269301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a007ffa-5f34-4f21-98dd-5929712d66d0 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:22.281529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 109/222] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:22.286710 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a3e9e-b22c-4c3b-b358-64ee6d1d3b5f None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:22.286710 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a3e9e-b22c-4c3b-b358-64ee6d1d3b5f None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:22.286710 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a3e9e-b22c-4c3b-b358-64ee6d1d3b5f None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:22.286710 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 110/223] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:22.289413 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a3e9e-b22c-4c3b-b358-64ee6d1d3b5f None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:22.309063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828d80d-9c8e-4946-aca0-59c943e62486 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:22.362109 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c2239f-6152-45f3-9a71-c5dbae5c7ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:22.362109 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2239f-6152-45f3-9a71-c5dbae5c7ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:22.362109 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2239f-6152-45f3-9a71-c5dbae5c7ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:22.362109 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c2239f-6152-45f3-9a71-c5dbae5c7ac9 None admin] PATH_INFO: `/v3/projects/f8bf83f1361247688208d846845c9d82` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:22.362517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2239f-6152-45f3-9a71-c5dbae5c7ac9 None admin] RBAC: Authorizing `identity:get_project(project_id=f8bf83f1361247688208d846845c9d82)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:22.364310 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c2239f-6152-45f3-9a71-c5dbae5c7ac9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:22.367351 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 114/224] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:22 2022] GET /identity/v3/projects/f8bf83f1361247688208d846845c9d82 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:22.368550 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6ba12-ee65-4e8b-9436-eb4c8ff26c2a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:22.368834 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6ba12-ee65-4e8b-9436-eb4c8ff26c2a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:22.369107 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6ba12-ee65-4e8b-9436-eb4c8ff26c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:22.391646 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5da6ba12-ee65-4e8b-9436-eb4c8ff26c2a None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:22.392798 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5da6ba12-ee65-4e8b-9436-eb4c8ff26c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:22.404007 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:22.431416 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a3e9e-b22c-4c3b-b358-64ee6d1d3b5f None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:22.478209 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4efa60-a511-48ed-b130-74cec85886a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:22.479147 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4efa60-a511-48ed-b130-74cec85886a2 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:22.479570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4efa60-a511-48ed-b130-74cec85886a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:22.480014 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4efa60-a511-48ed-b130-74cec85886a2 None admin] PATH_INFO: `/v3/users/430270d2a8664baab8b8a0415d1dbb24` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:22.481996 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 115/225] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:22.483001 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6ba12-ee65-4e8b-9436-eb4c8ff26c2a None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:22.487068 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4efa60-a511-48ed-b130-74cec85886a2 None admin] RBAC: Authorizing `identity:get_user(user_id=430270d2a8664baab8b8a0415d1dbb24)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:22.490163 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4efa60-a511-48ed-b130-74cec85886a2 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:22.493046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 111/226] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:22 2022] GET /identity/v3/users/430270d2a8664baab8b8a0415d1dbb24 => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:22.498582 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4efa60-a511-48ed-b130-74cec85886a2 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:22.559381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:22.604505 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6ba12-ee65-4e8b-9436-eb4c8ff26c2a None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:22.608871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4efa60-a511-48ed-b130-74cec85886a2 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:22.643379 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47dd09-2550-42af-b2ae-5df6180b7283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:22.644149 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47dd09-2550-42af-b2ae-5df6180b7283 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:22.644313 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47dd09-2550-42af-b2ae-5df6180b7283 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:22.644590 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47dd09-2550-42af-b2ae-5df6180b7283 None admin] PATH_INFO: `/v3/projects/f8bf83f1361247688208d846845c9d82` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:22.645751 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47dd09-2550-42af-b2ae-5df6180b7283 None admin] RBAC: Authorizing `identity:get_project(project_id=f8bf83f1361247688208d846845c9d82)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:22.647370 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47dd09-2550-42af-b2ae-5df6180b7283 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:22.649805 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 112/227] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:22 2022] GET /identity/v3/projects/f8bf83f1361247688208d846845c9d82 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:22.650171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdb559-592a-4d33-86ae-e1e114a7ee9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:22.650607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47dd09-2550-42af-b2ae-5df6180b7283 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:22.651270 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdb559-592a-4d33-86ae-e1e114a7ee9f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:22.651859 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdb559-592a-4d33-86ae-e1e114a7ee9f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:22.652305 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdb559-592a-4d33-86ae-e1e114a7ee9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:22.653384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cdb559-592a-4d33-86ae-e1e114a7ee9f None admin] RBAC: Adding query filter params (role.id=68fe920c9be84b3aa1d35bdce554d96a, scope.project.id=f8bf83f1361247688208d846845c9d82, user.id=1394e6b314554594bf617f454f136113) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:22.653869 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cdb559-592a-4d33-86ae-e1e114a7ee9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:22.655568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cdb559-592a-4d33-86ae-e1e114a7ee9f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:22.661986 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 116/228] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:46:22 2022] GET /identity/v3/role_assignments?user.id=1394e6b314554594bf617f454f136113&scope.project.id=f8bf83f1361247688208d846845c9d82&role.id=68fe920c9be84b3aa1d35bdce554d96a => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:22.662999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdb559-592a-4d33-86ae-e1e114a7ee9f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:22.670184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:22.763976 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47dd09-2550-42af-b2ae-5df6180b7283 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:22.781561 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdb559-592a-4d33-86ae-e1e114a7ee9f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:22.805591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e28e8-2779-40d6-92fe-7133db9da775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:22.806278 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e28e8-2779-40d6-92fe-7133db9da775 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:22.806560 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e28e8-2779-40d6-92fe-7133db9da775 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:22.806832 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e28e8-2779-40d6-92fe-7133db9da775 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:22.815699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:22.824066 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9e28e8-2779-40d6-92fe-7133db9da775 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:22.829139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-2c9e28e8-2779-40d6-92fe-7133db9da775 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:22.829139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:22.829139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:22.829139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:22.829139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:22.829139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:22.829139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:22.829139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:22.829139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:22.829139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:22.829139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:22.829139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:22.829139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:22.829139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:22.829139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:22.829139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:22.830119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:22.830119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:22.830119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:22.830119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:22.830119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:22.830119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:22.830119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:22.830119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:22.830119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:22.830119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:22.830119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:22.830119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:22.830119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:22.830119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:22.830119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:22.830119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:22.830930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:22.830930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:22.830930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:22.830930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:22.830930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:22.830930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:22.830930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:22.830930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:22.830930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:22.830930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:22.830930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:22.830930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:22.830930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 113/229] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 2 08:46:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:22.833685 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8e20a-3648-4ba6-a0e7-4d24cad8370d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:22.834696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8e20a-3648-4ba6-a0e7-4d24cad8370d None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:22.835193 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8e20a-3648-4ba6-a0e7-4d24cad8370d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:22.835791 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8e20a-3648-4ba6-a0e7-4d24cad8370d None admin] PATH_INFO: `/v3/projects/f8bf83f1361247688208d846845c9d82/users/430270d2a8664baab8b8a0415d1dbb24/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:22.836612 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e28e8-2779-40d6-92fe-7133db9da775 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:22.838292 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8e20a-3648-4ba6-a0e7-4d24cad8370d None admin] RBAC: Authorizing `identity:create_grant(project_id=f8bf83f1361247688208d846845c9d82, user_id=430270d2a8664baab8b8a0415d1dbb24, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:22.840078 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8e20a-3648-4ba6-a0e7-4d24cad8370d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:22.921128 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:22.944772 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e28e8-2779-40d6-92fe-7133db9da775 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:22.958973 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 117/230] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:46:22 2022] PUT /identity/v3/projects/f8bf83f1361247688208d846845c9d82/users/430270d2a8664baab8b8a0415d1dbb24/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:46:22.984439 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90073809-9efa-40d4-bfd6-33922c300b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:22.985094 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90073809-9efa-40d4-bfd6-33922c300b96 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:22.985363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90073809-9efa-40d4-bfd6-33922c300b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:22.985640 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90073809-9efa-40d4-bfd6-33922c300b96 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:22.986427 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90073809-9efa-40d4-bfd6-33922c300b96 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:22.986738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90073809-9efa-40d4-bfd6-33922c300b96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:22.988171 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90073809-9efa-40d4-bfd6-33922c300b96 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:22.998427 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 114/231] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:46:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:23.004352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8e20a-3648-4ba6-a0e7-4d24cad8370d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:23.161456 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8e20a-3648-4ba6-a0e7-4d24cad8370d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:23.199377 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409a112-048c-4237-9034-2570aa3947ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:23.200015 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409a112-048c-4237-9034-2570aa3947ab None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:23.201135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409a112-048c-4237-9034-2570aa3947ab None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:23.201135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5409a112-048c-4237-9034-2570aa3947ab None admin] PATH_INFO: `/v3/users/430270d2a8664baab8b8a0415d1dbb24` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:23.201765 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409a112-048c-4237-9034-2570aa3947ab None admin] RBAC: Authorizing `identity:get_user(user_id=430270d2a8664baab8b8a0415d1dbb24)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:23.203295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5409a112-048c-4237-9034-2570aa3947ab None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:23.205175 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 118/232] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:23 2022] GET /identity/v3/users/430270d2a8664baab8b8a0415d1dbb24 => generated 283 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:23.210792 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90073809-9efa-40d4-bfd6-33922c300b96 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:23.320772 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90073809-9efa-40d4-bfd6-33922c300b96 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:23.357683 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d61f0e-e2bb-479d-afcb-6800d7e1afe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:23.358387 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d61f0e-e2bb-479d-afcb-6800d7e1afe8 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:23.358669 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d61f0e-e2bb-479d-afcb-6800d7e1afe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:23.358948 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d61f0e-e2bb-479d-afcb-6800d7e1afe8 None admin] PATH_INFO: `/v3/projects/f8bf83f1361247688208d846845c9d82` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:23.360217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d61f0e-e2bb-479d-afcb-6800d7e1afe8 None admin] RBAC: Authorizing `identity:get_project(project_id=f8bf83f1361247688208d846845c9d82)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:23.361713 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d61f0e-e2bb-479d-afcb-6800d7e1afe8 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:23.363552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 115/233] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:23 2022] GET /identity/v3/projects/f8bf83f1361247688208d846845c9d82 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:23.370083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409a112-048c-4237-9034-2570aa3947ab None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:23.422521 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:23.492559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5409a112-048c-4237-9034-2570aa3947ab None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:23.531814 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b29f23e-0408-4c20-9180-d1d6f5ab2946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:23.532488 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29f23e-0408-4c20-9180-d1d6f5ab2946 None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:23.532771 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29f23e-0408-4c20-9180-d1d6f5ab2946 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:23.533054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b29f23e-0408-4c20-9180-d1d6f5ab2946 None admin] PATH_INFO: `/v3/projects/f8bf83f1361247688208d846845c9d82/users/430270d2a8664baab8b8a0415d1dbb24/roles/111e800ddb0a411c80664cd51ce13fc6` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:23.535309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b29f23e-0408-4c20-9180-d1d6f5ab2946 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8bf83f1361247688208d846845c9d82, user_id=430270d2a8664baab8b8a0415d1dbb24, role_id=111e800ddb0a411c80664cd51ce13fc6)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:23.536762 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b29f23e-0408-4c20-9180-d1d6f5ab2946 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:23.645787 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 119/234] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:46:23 2022] PUT /identity/v3/projects/f8bf83f1361247688208d846845c9d82/users/430270d2a8664baab8b8a0415d1dbb24/roles/111e800ddb0a411c80664cd51ce13fc6 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:46:24.256010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07593ba-987d-41a6-b9bc-42f581d5845d None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:24.256513 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07593ba-987d-41a6-b9bc-42f581d5845d None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:24.256513 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07593ba-987d-41a6-b9bc-42f581d5845d None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:24.258490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 116/235] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:24.263642 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0899f7a-07c7-496a-acd2-930607e5aa24 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:24.264057 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0899f7a-07c7-496a-acd2-930607e5aa24 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:24.264229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0899f7a-07c7-496a-acd2-930607e5aa24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:24.290505 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-e0899f7a-07c7-496a-acd2-930607e5aa24 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:24.291704 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e0899f7a-07c7-496a-acd2-930607e5aa24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:24.389979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 120/236] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:24.424473 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07593ba-987d-41a6-b9bc-42f581d5845d None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:24.435241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:24.451404 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52b36b-853a-4ad9-aa2b-75eac86612d8 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:24.451828 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52b36b-853a-4ad9-aa2b-75eac86612d8 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:24.451965 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52b36b-853a-4ad9-aa2b-75eac86612d8 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:24.453579 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 121/237] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:24.458210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfe4c7-13ed-4c27-a423-93b89a7b70f2 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:24.458499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfe4c7-13ed-4c27-a423-93b89a7b70f2 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:24.458778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfe4c7-13ed-4c27-a423-93b89a7b70f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:24.480177 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-94cfe4c7-13ed-4c27-a423-93b89a7b70f2 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:24.481370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-94cfe4c7-13ed-4c27-a423-93b89a7b70f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:24.524330 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:24.564048 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 122/238] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:24.602262 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07593ba-987d-41a6-b9bc-42f581d5845d None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:24.619281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfe4c7-13ed-4c27-a423-93b89a7b70f2 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:24.643708 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01826ab-3169-468b-adf4-d13b02192aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:24.644372 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01826ab-3169-468b-adf4-d13b02192aef None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:24.644651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01826ab-3169-468b-adf4-d13b02192aef None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:24.644927 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01826ab-3169-468b-adf4-d13b02192aef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:24.665419 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01826ab-3169-468b-adf4-d13b02192aef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:24.672344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-c01826ab-3169-468b-adf4-d13b02192aef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:46:24.672344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:24.672344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:24.672344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:24.672344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:24.672344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:24.672344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:24.672344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:24.672344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:24.672344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:24.672344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:24.672344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:24.672344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:24.672344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:24.672344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:24.672344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:24.673215 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:24.673215 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:24.673215 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:24.673215 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:24.673215 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:24.673215 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:24.673215 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:24.673215 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:24.673215 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:24.673215 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:24.673215 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:24.673215 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:24.673215 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:24.673215 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:24.673215 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:24.673215 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:24.674050 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:24.674050 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:24.674050 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:24.674050 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:24.674050 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:24.674050 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:24.674050 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:24.674050 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:24.674050 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:24.674050 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:24.674050 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:46:24.674050 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:24.674651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 117/239] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:24.680970 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01826ab-3169-468b-adf4-d13b02192aef None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:24.807993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfe4c7-13ed-4c27-a423-93b89a7b70f2 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:24.812263 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:24.858781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01826ab-3169-468b-adf4-d13b02192aef None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:24.897660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beeea1e-0bee-4d46-a5d4-83b99fb58cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:24.898858 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beeea1e-0bee-4d46-a5d4-83b99fb58cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:24.899143 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beeea1e-0bee-4d46-a5d4-83b99fb58cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:24.899413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beeea1e-0bee-4d46-a5d4-83b99fb58cf5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:24.900569 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beeea1e-0bee-4d46-a5d4-83b99fb58cf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:24.902105 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beeea1e-0bee-4d46-a5d4-83b99fb58cf5 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:24.903722 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 123/240] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:46:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:24.904589 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a3f82-9499-4e51-a6ae-be85f4255a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:24.905487 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a3f82-9499-4e51-a6ae-be85f4255a8a None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:24.906021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a3f82-9499-4e51-a6ae-be85f4255a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:24.906500 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4a3f82-9499-4e51-a6ae-be85f4255a8a None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:24.907414 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a3f82-9499-4e51-a6ae-be85f4255a8a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:24.911206 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beeea1e-0bee-4d46-a5d4-83b99fb58cf5 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:24.911607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a3f82-9499-4e51-a6ae-be85f4255a8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:24.913272 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4a3f82-9499-4e51-a6ae-be85f4255a8a None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:24.943519 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 118/241] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:24.949388 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a3f82-9499-4e51-a6ae-be85f4255a8a None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:24.954874 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:25.053001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4a3f82-9499-4e51-a6ae-be85f4255a8a None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:25.077309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beeea1e-0bee-4d46-a5d4-83b99fb58cf5 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:25.081310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:25.090269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6182c06-d945-453a-9e86-5197ea0955fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:25.090973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6182c06-d945-453a-9e86-5197ea0955fd None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:25.091255 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6182c06-d945-453a-9e86-5197ea0955fd None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:25.101317 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6182c06-d945-453a-9e86-5197ea0955fd None admin] PATH_INFO: `/v3/users/8db4498da11d43959b4197493394e55a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:25.106234 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6182c06-d945-453a-9e86-5197ea0955fd None admin] RBAC: Authorizing `identity:get_user(user_id=8db4498da11d43959b4197493394e55a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:25.107710 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6182c06-d945-453a-9e86-5197ea0955fd None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:25.109808 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 119/242] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:24 2022] GET /identity/v3/users/8db4498da11d43959b4197493394e55a => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:25.116239 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6182c06-d945-453a-9e86-5197ea0955fd None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:25.145066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd2f30f-a092-455f-934a-8fa118eb0923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:25.146063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2f30f-a092-455f-934a-8fa118eb0923 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:25.146548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2f30f-a092-455f-934a-8fa118eb0923 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:25.147042 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2f30f-a092-455f-934a-8fa118eb0923 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:25.149688 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd2f30f-a092-455f-934a-8fa118eb0923 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:25.151387 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd2f30f-a092-455f-934a-8fa118eb0923 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:25.239087 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 124/243] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 2 08:46:24 2022] POST /identity/v3/users => generated 293 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 08:46:25.255841 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6182c06-d945-453a-9e86-5197ea0955fd None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:25.274163 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cd619-f20b-407d-ba5e-8a7fd843ea7d None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:25.275151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cd619-f20b-407d-ba5e-8a7fd843ea7d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:25.275654 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cd619-f20b-407d-ba5e-8a7fd843ea7d None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:25.279734 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 125/244] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:25.284922 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08fe79-a941-4d1e-b4d3-d9a4d7ee2367 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:25.285443 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08fe79-a941-4d1e-b4d3-d9a4d7ee2367 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:25.285899 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08fe79-a941-4d1e-b4d3-d9a4d7ee2367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:25.296338 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a26d0-96dc-443c-a1ed-148c71e8636e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:25.296974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a26d0-96dc-443c-a1ed-148c71e8636e None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:25.297259 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a26d0-96dc-443c-a1ed-148c71e8636e None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:25.297536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62a26d0-96dc-443c-a1ed-148c71e8636e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:25.320779 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-1e08fe79-a941-4d1e-b4d3-d9a4d7ee2367 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:25.322375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1e08fe79-a941-4d1e-b4d3-d9a4d7ee2367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:25.366519 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:25.381611 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a26d0-96dc-443c-a1ed-148c71e8636e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:25.383400 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62a26d0-96dc-443c-a1ed-148c71e8636e None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:25.501620 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 120/245] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 08:46:25.507339 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a26d0-96dc-443c-a1ed-148c71e8636e None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:25.513191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 126/246] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:25.535316 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08fe79-a941-4d1e-b4d3-d9a4d7ee2367 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:25.576816 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:25.610139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:25.627846 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62a26d0-96dc-443c-a1ed-148c71e8636e None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:25.674766 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4073a-dd5a-4156-823f-378064a29f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:25.675462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4073a-dd5a-4156-823f-378064a29f16 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:25.675750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4073a-dd5a-4156-823f-378064a29f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:25.676031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4073a-dd5a-4156-823f-378064a29f16 None admin] PATH_INFO: `/v3/projects/f8bf83f1361247688208d846845c9d82` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:25.677749 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4073a-dd5a-4156-823f-378064a29f16 None admin] RBAC: Authorizing `identity:get_project(project_id=f8bf83f1361247688208d846845c9d82)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:25.680182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4073a-dd5a-4156-823f-378064a29f16 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:25.682884 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 121/247] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:25 2022] GET /identity/v3/projects/f8bf83f1361247688208d846845c9d82 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:25.684036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1291a-7338-42c6-a1f5-2387d4fb6f62 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:25.684337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1291a-7338-42c6-a1f5-2387d4fb6f62 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:25.684617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1291a-7338-42c6-a1f5-2387d4fb6f62 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:25.691228 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 122/248] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:25.692039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08fe79-a941-4d1e-b4d3-d9a4d7ee2367 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:25.693003 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1291a-7338-42c6-a1f5-2387d4fb6f62 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:25.748808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6d4f0-531e-4d63-ba1b-a9a57bbfd9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:25.749608 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6d4f0-531e-4d63-ba1b-a9a57bbfd9bf None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:25.749919 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6d4f0-531e-4d63-ba1b-a9a57bbfd9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:25.750402 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6d4f0-531e-4d63-ba1b-a9a57bbfd9bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:25.763126 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6d4f0-531e-4d63-ba1b-a9a57bbfd9bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:25.768525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-9ef6d4f0-531e-4d63-ba1b-a9a57bbfd9bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:25.768525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:25.768525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:25.768525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:25.768525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:25.768525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:25.768525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:25.768525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:25.768525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:25.768525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:25.768525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:25.768525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:25.768525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:25.768525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:25.768525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:25.768525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:25.769421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:25.769421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:25.769421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:25.769421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:25.769421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:25.769421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:25.769421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:25.769421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:25.769421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:25.769421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:25.769421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:25.769421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:25.769421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:25.769421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:25.769421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:25.769421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:25.770283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:25.770283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:25.770283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:25.770283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:25.770283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:25.770283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:25.770283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:25.770283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:25.770283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:25.770283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:25.770283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:25.770283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:25.770283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 127/249] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:25.771478 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab375dd1-d857-4b71-b897-21de1492cb42 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:25.771652 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab375dd1-d857-4b71-b897-21de1492cb42 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:25.771901 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab375dd1-d857-4b71-b897-21de1492cb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:25.792500 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-ab375dd1-d857-4b71-b897-21de1492cb42 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:25.793716 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ab375dd1-d857-4b71-b897-21de1492cb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:25.823523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1291a-7338-42c6-a1f5-2387d4fb6f62 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:25.903155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b2c00-77fa-40d3-bc29-bf59d7822483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:25.904096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b2c00-77fa-40d3-bc29-bf59d7822483 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:25.905076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b2c00-77fa-40d3-bc29-bf59d7822483 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:25.905875 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b2c00-77fa-40d3-bc29-bf59d7822483 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:25.907179 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b2c00-77fa-40d3-bc29-bf59d7822483 None admin] RBAC: Adding query filter params (role.id=3c439be3950947478195dcc08835a9b5, scope.project.id=f8bf83f1361247688208d846845c9d82, user.id=8db4498da11d43959b4197493394e55a) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:25.907680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b2c00-77fa-40d3-bc29-bf59d7822483 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:25.909777 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b2c00-77fa-40d3-bc29-bf59d7822483 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:25.918568 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 123/250] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:46:25 2022] GET /identity/v3/role_assignments?user.id=8db4498da11d43959b4197493394e55a&scope.project.id=f8bf83f1361247688208d846845c9d82&role.id=3c439be3950947478195dcc08835a9b5 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:25.919815 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b2c00-77fa-40d3-bc29-bf59d7822483 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:25.927278 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 128/251] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:25.956275 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:25.979287 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab375dd1-d857-4b71-b897-21de1492cb42 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:26.028030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:26.076105 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b2c00-77fa-40d3-bc29-bf59d7822483 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:26.119979 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d628816-f500-4201-9954-ba041abca04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:26.121389 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d628816-f500-4201-9954-ba041abca04a None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:26.122085 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d628816-f500-4201-9954-ba041abca04a None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:26.122545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d628816-f500-4201-9954-ba041abca04a None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:26.124043 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d628816-f500-4201-9954-ba041abca04a None admin] RBAC: Adding query filter params (name=member) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:26.124553 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d628816-f500-4201-9954-ba041abca04a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:26.126334 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d628816-f500-4201-9954-ba041abca04a None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:26.137363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 124/252] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:26.142916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d628816-f500-4201-9954-ba041abca04a None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:26.262399 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab375dd1-d857-4b71-b897-21de1492cb42 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:26.278411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:26.327779 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d628816-f500-4201-9954-ba041abca04a None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:26.348195 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596c0f9-f3c6-440e-8600-b7dd1da2385d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:26.348875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596c0f9-f3c6-440e-8600-b7dd1da2385d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:26.349129 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596c0f9-f3c6-440e-8600-b7dd1da2385d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:26.349402 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596c0f9-f3c6-440e-8600-b7dd1da2385d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:26.372430 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1596c0f9-f3c6-440e-8600-b7dd1da2385d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:26.377811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-1596c0f9-f3c6-440e-8600-b7dd1da2385d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:46:26.377811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:26.377811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:26.377811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:26.377811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:26.377811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:26.377811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:26.377811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:26.377811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:26.377811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:26.377811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:26.377811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:26.377811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:26.377811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:26.377811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:26.377811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:26.378719 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:26.378719 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:26.378719 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:26.378719 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:26.378719 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:26.378719 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:26.378719 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:26.378719 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:26.378719 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:26.378719 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:26.378719 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:26.378719 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:26.378719 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:26.378719 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:26.378719 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:26.378719 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:26.379517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:26.379517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:26.379517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:26.379517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:26.379517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:26.379517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:26.379517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:26.379517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:26.379517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:26.379517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:26.379517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:46:26.379517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:26.384961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 129/253] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:46:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:26.391617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db449149-5ce6-41c1-99b7-a33a947cb7c9 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:26.391898 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db449149-5ce6-41c1-99b7-a33a947cb7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:26.392169 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db449149-5ce6-41c1-99b7-a33a947cb7c9 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:26.393487 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 130/254] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:26.397518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db449149-5ce6-41c1-99b7-a33a947cb7c9 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:26.405086 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906357fc-aae0-4fa3-b887-54b5faefdb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:26.405736 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906357fc-aae0-4fa3-b887-54b5faefdb61 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:26.406037 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906357fc-aae0-4fa3-b887-54b5faefdb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:26.406310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906357fc-aae0-4fa3-b887-54b5faefdb61 None admin] PATH_INFO: `/v3/users/1394e6b314554594bf617f454f136113` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:26.411665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906357fc-aae0-4fa3-b887-54b5faefdb61 None admin] RBAC: Authorizing `identity:get_user(user_id=1394e6b314554594bf617f454f136113)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:26.413114 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906357fc-aae0-4fa3-b887-54b5faefdb61 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:26.415346 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 125/255] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:26 2022] GET /identity/v3/users/1394e6b314554594bf617f454f136113 => generated 297 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:26.418485 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83345a-b0ed-459f-8eb9-5b7e2be24e01 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:26.418925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83345a-b0ed-459f-8eb9-5b7e2be24e01 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:26.419205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83345a-b0ed-459f-8eb9-5b7e2be24e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:26.450435 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-0e83345a-b0ed-459f-8eb9-5b7e2be24e01 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:26.452793 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-0e83345a-b0ed-459f-8eb9-5b7e2be24e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:26.534373 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:26.587739 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 126/256] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:26.588610 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83345a-b0ed-459f-8eb9-5b7e2be24e01 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:26.598793 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db449149-5ce6-41c1-99b7-a33a947cb7c9 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:26.676444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b76d8e-2339-4e10-b0b5-defc1c0373a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:26.677139 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b76d8e-2339-4e10-b0b5-defc1c0373a1 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:26.677424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b76d8e-2339-4e10-b0b5-defc1c0373a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:26.677702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b76d8e-2339-4e10-b0b5-defc1c0373a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:26.678513 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b76d8e-2339-4e10-b0b5-defc1c0373a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:26.678826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b76d8e-2339-4e10-b0b5-defc1c0373a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:26.680288 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b76d8e-2339-4e10-b0b5-defc1c0373a1 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:26.697737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 131/257] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:26.701979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b76d8e-2339-4e10-b0b5-defc1c0373a1 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:26.727608 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:26.767731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83345a-b0ed-459f-8eb9-5b7e2be24e01 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:26.836850 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652c320-bfeb-49c8-b96b-7b318bf46323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:26.837426 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652c320-bfeb-49c8-b96b-7b318bf46323 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:26.837836 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652c320-bfeb-49c8-b96b-7b318bf46323 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:26.838059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8652c320-bfeb-49c8-b96b-7b318bf46323 None admin] PATH_INFO: `/v3/projects/f8bf83f1361247688208d846845c9d82` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:26.842630 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652c320-bfeb-49c8-b96b-7b318bf46323 None admin] RBAC: Authorizing `identity:get_project(project_id=f8bf83f1361247688208d846845c9d82)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:26.844110 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8652c320-bfeb-49c8-b96b-7b318bf46323 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:26.846111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 127/258] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:26 2022] GET /identity/v3/projects/f8bf83f1361247688208d846845c9d82 => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:26.846594 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652c320-bfeb-49c8-b96b-7b318bf46323 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:26.850289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b76d8e-2339-4e10-b0b5-defc1c0373a1 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:26.887573 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a39cf-3339-41a0-8a88-bb163d9c1c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:26.888612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a39cf-3339-41a0-8a88-bb163d9c1c46 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:26.889201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a39cf-3339-41a0-8a88-bb163d9c1c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:26.889682 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2a39cf-3339-41a0-8a88-bb163d9c1c46 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:26.901075 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2a39cf-3339-41a0-8a88-bb163d9c1c46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:26.906999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-1c2a39cf-3339-41a0-8a88-bb163d9c1c46 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:26.906999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:26.906999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:26.906999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:26.906999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:26.906999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:26.906999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:26.906999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:26.906999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:26.906999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:26.906999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:26.906999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:26.906999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:26.906999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:26.906999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:26.906999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:26.907886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:26.907886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:26.907886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:26.907886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:26.907886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:26.907886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:26.907886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:26.907886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:26.907886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:26.907886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:26.907886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:26.907886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:26.907886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:26.907886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:26.907886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:26.907886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:26.908748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:26.908748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:26.908748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:26.908748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:26.908748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:26.908748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:26.908748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:26.908748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:26.908748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:26.908748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:26.908748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:26.908748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:26.911117 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 132/259] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 2 08:46:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:26.912089 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a39cf-3339-41a0-8a88-bb163d9c1c46 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:26.917491 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:27.061498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8652c320-bfeb-49c8-b96b-7b318bf46323 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:27.112829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2a39cf-3339-41a0-8a88-bb163d9c1c46 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:27.128425 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc94744-00a3-4798-b115-bf49b7f6b5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:27.129083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc94744-00a3-4798-b115-bf49b7f6b5bd None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:27.129368 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc94744-00a3-4798-b115-bf49b7f6b5bd None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:27.129650 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc94744-00a3-4798-b115-bf49b7f6b5bd None admin] PATH_INFO: `/v3/users/430270d2a8664baab8b8a0415d1dbb24` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:27.132901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc94744-00a3-4798-b115-bf49b7f6b5bd None admin] RBAC: Authorizing `identity:get_user(user_id=430270d2a8664baab8b8a0415d1dbb24)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:27.134394 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc94744-00a3-4798-b115-bf49b7f6b5bd None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:27.136383 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 128/260] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:26 2022] GET /identity/v3/users/430270d2a8664baab8b8a0415d1dbb24 => generated 283 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:27.137392 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc94744-00a3-4798-b115-bf49b7f6b5bd None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:27.157751 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:27.228646 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e64c3c-0906-43df-9d5d-4bee624394f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:27.229311 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e64c3c-0906-43df-9d5d-4bee624394f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:27.229604 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e64c3c-0906-43df-9d5d-4bee624394f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:27.229900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e64c3c-0906-43df-9d5d-4bee624394f5 None admin] PATH_INFO: `/v3/projects/f8bf83f1361247688208d846845c9d82/users/1394e6b314554594bf617f454f136113/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:27.235293 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e64c3c-0906-43df-9d5d-4bee624394f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8bf83f1361247688208d846845c9d82, user_id=1394e6b314554594bf617f454f136113, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:27.236728 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e64c3c-0906-43df-9d5d-4bee624394f5 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:27.255453 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc94744-00a3-4798-b115-bf49b7f6b5bd None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:27.293551 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823c62b-35d0-4525-896f-1f005468bc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:27.294497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823c62b-35d0-4525-896f-1f005468bc04 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:27.294932 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823c62b-35d0-4525-896f-1f005468bc04 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:27.295358 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823c62b-35d0-4525-896f-1f005468bc04 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:27.296319 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5823c62b-35d0-4525-896f-1f005468bc04 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:27.296804 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5823c62b-35d0-4525-896f-1f005468bc04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:27.298498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5823c62b-35d0-4525-896f-1f005468bc04 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:27.309485 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 129/261] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:46:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:27.310777 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823c62b-35d0-4525-896f-1f005468bc04 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:27.342369 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 133/262] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:46:26 2022] PUT /identity/v3/projects/f8bf83f1361247688208d846845c9d82/users/1394e6b314554594bf617f454f136113/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:46:27.343233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e64c3c-0906-43df-9d5d-4bee624394f5 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:27.400069 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:27.439682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823c62b-35d0-4525-896f-1f005468bc04 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:27.493348 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:27.526627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63ee73-119b-4420-8ae3-3f8de4949d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:27.527635 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63ee73-119b-4420-8ae3-3f8de4949d78 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:27.528124 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63ee73-119b-4420-8ae3-3f8de4949d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:27.528683 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63ee73-119b-4420-8ae3-3f8de4949d78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:27.604844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e64c3c-0906-43df-9d5d-4bee624394f5 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:27.672319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0b447-07a8-4101-aca7-522292017eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:27.672981 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0b447-07a8-4101-aca7-522292017eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:27.673477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0b447-07a8-4101-aca7-522292017eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:27.673759 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d0b447-07a8-4101-aca7-522292017eb2 None admin] PATH_INFO: `/v3/users/430270d2a8664baab8b8a0415d1dbb24` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:27.677203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d0b447-07a8-4101-aca7-522292017eb2 None admin] RBAC: Authorizing `identity:get_user(user_id=430270d2a8664baab8b8a0415d1dbb24)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:27.678699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d0b447-07a8-4101-aca7-522292017eb2 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:27.680924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 134/263] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:27 2022] GET /identity/v3/users/430270d2a8664baab8b8a0415d1dbb24 => generated 283 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:27.686287 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63ee73-119b-4420-8ae3-3f8de4949d78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:27.686442 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0b447-07a8-4101-aca7-522292017eb2 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:27.687775 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63ee73-119b-4420-8ae3-3f8de4949d78 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:27.807919 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d0b447-07a8-4101-aca7-522292017eb2 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:27.836156 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:27.848260 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb0c87-cb0c-45ed-9297-653c475a6c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:27.848947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb0c87-cb0c-45ed-9297-653c475a6c03 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:27.849355 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb0c87-cb0c-45ed-9297-653c475a6c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:27.849637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb0c87-cb0c-45ed-9297-653c475a6c03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:27.886629 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:27.932394 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb0c87-cb0c-45ed-9297-653c475a6c03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:27.933827 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb0c87-cb0c-45ed-9297-653c475a6c03 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:27.940579 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 130/264] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:46:27.948049 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63ee73-119b-4420-8ae3-3f8de4949d78 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:28.041849 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 135/265] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 08:46:28.047552 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb0c87-cb0c-45ed-9297-653c475a6c03 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:28.105782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63ee73-119b-4420-8ae3-3f8de4949d78 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:28.106917 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:28.144940 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685dddc-d077-4370-aeb0-8655dfc38f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:28.145999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685dddc-d077-4370-aeb0-8655dfc38f53 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:28.146490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685dddc-d077-4370-aeb0-8655dfc38f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:28.146992 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685dddc-d077-4370-aeb0-8655dfc38f53 None admin] PATH_INFO: `/v3/projects/f8bf83f1361247688208d846845c9d82` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:28.148994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685dddc-d077-4370-aeb0-8655dfc38f53 None admin] RBAC: Authorizing `identity:get_project(project_id=f8bf83f1361247688208d846845c9d82)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:28.150714 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d685dddc-d077-4370-aeb0-8655dfc38f53 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:28.153129 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 131/266] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:27 2022] GET /identity/v3/projects/f8bf83f1361247688208d846845c9d82 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:28.161653 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685dddc-d077-4370-aeb0-8655dfc38f53 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:28.230743 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:28.271621 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb0c87-cb0c-45ed-9297-653c475a6c03 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:28.319603 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d685dddc-d077-4370-aeb0-8655dfc38f53 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:28.339438 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f652ef8a-a540-4df8-b772-a28ebb395596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:28.340134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652ef8a-a540-4df8-b772-a28ebb395596 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:28.340365 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652ef8a-a540-4df8-b772-a28ebb395596 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:28.340638 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f652ef8a-a540-4df8-b772-a28ebb395596 None admin] PATH_INFO: `/v3/projects/f8bf83f1361247688208d846845c9d82` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:28.341804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652ef8a-a540-4df8-b772-a28ebb395596 None admin] RBAC: Authorizing `identity:get_project(project_id=f8bf83f1361247688208d846845c9d82)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:28.343497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f652ef8a-a540-4df8-b772-a28ebb395596 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:28.345399 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 136/267] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:28 2022] GET /identity/v3/projects/f8bf83f1361247688208d846845c9d82 => generated 302 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:28.346514 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93311646-059f-4e80-829e-09b3b9bf7dbb None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:28.346799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93311646-059f-4e80-829e-09b3b9bf7dbb None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:28.347080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93311646-059f-4e80-829e-09b3b9bf7dbb None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:28.348518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 137/268] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:28.349513 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c8b99-15ea-4474-95be-79909970654f None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:28.349810 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c8b99-15ea-4474-95be-79909970654f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:28.350198 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c8b99-15ea-4474-95be-79909970654f None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:28.352071 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 138/269] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:28.356888 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c8b99-15ea-4474-95be-79909970654f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:28.366162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b608fc7c-705e-47b8-bf7b-95cc490bf97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:28.367710 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608fc7c-705e-47b8-bf7b-95cc490bf97c None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:28.368326 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608fc7c-705e-47b8-bf7b-95cc490bf97c None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:28.369178 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b608fc7c-705e-47b8-bf7b-95cc490bf97c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:28.370528 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608fc7c-705e-47b8-bf7b-95cc490bf97c None admin] RBAC: Adding query filter params (role.id=8ce0850e15804f3d811e88cc9841e673, scope.project.id=f8bf83f1361247688208d846845c9d82, user.id=430270d2a8664baab8b8a0415d1dbb24) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:28.371055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608fc7c-705e-47b8-bf7b-95cc490bf97c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:28.372704 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b608fc7c-705e-47b8-bf7b-95cc490bf97c None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:28.381046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 132/270] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:46:28 2022] GET /identity/v3/role_assignments?user.id=430270d2a8664baab8b8a0415d1dbb24&scope.project.id=f8bf83f1361247688208d846845c9d82&role.id=8ce0850e15804f3d811e88cc9841e673 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:28.382752 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063c983-b27b-40c8-90d9-2158cc071168 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:28.383250 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063c983-b27b-40c8-90d9-2158cc071168 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:28.383731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063c983-b27b-40c8-90d9-2158cc071168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:28.407618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-a063c983-b27b-40c8-90d9-2158cc071168 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:28.410246 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-a063c983-b27b-40c8-90d9-2158cc071168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:28.467945 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:28.481047 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c8b99-15ea-4474-95be-79909970654f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:28.509849 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 133/271] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:28.512455 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6013853-a7fe-491d-88d2-65df50313e7d None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:28.513249 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6013853-a7fe-491d-88d2-65df50313e7d None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:28.513712 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6013853-a7fe-491d-88d2-65df50313e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:28.522684 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f495f0d3-9b3b-4608-8b24-30784d95643c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:28.523344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495f0d3-9b3b-4608-8b24-30784d95643c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:28.523753 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495f0d3-9b3b-4608-8b24-30784d95643c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:28.524096 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495f0d3-9b3b-4608-8b24-30784d95643c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:28.525814 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f495f0d3-9b3b-4608-8b24-30784d95643c None admin] RBAC: Adding query filter params (role.id=111e800ddb0a411c80664cd51ce13fc6, scope.project.id=f8bf83f1361247688208d846845c9d82, user.id=430270d2a8664baab8b8a0415d1dbb24) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:28.525814 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f495f0d3-9b3b-4608-8b24-30784d95643c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:28.526930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f495f0d3-9b3b-4608-8b24-30784d95643c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:28.533867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 139/272] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:46:28 2022] GET /identity/v3/role_assignments?user.id=430270d2a8664baab8b8a0415d1dbb24&scope.project.id=f8bf83f1361247688208d846845c9d82&role.id=111e800ddb0a411c80664cd51ce13fc6 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:28.535488 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad063370-ade6-4f45-b420-ab46b8b2b6f0 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:28.535844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad063370-ade6-4f45-b420-ab46b8b2b6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:28.536189 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad063370-ade6-4f45-b420-ab46b8b2b6f0 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:28.537877 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 140/273] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:28.541025 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-e6013853-a7fe-491d-88d2-65df50313e7d None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:28.542620 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e6013853-a7fe-491d-88d2-65df50313e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:28.544800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6595c-23ef-473e-a0b4-f02cd975dd0c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:28.545159 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6595c-23ef-473e-a0b4-f02cd975dd0c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:28.545644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be6595c-23ef-473e-a0b4-f02cd975dd0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:28.575042 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5be6595c-23ef-473e-a0b4-f02cd975dd0c None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:28.577917 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5be6595c-23ef-473e-a0b4-f02cd975dd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:28.584687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:28.659049 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 134/274] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:28.665398 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6013853-a7fe-491d-88d2-65df50313e7d None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:28.721007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 141/275] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:28.724203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6595c-23ef-473e-a0b4-f02cd975dd0c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:28.804885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6013853-a7fe-491d-88d2-65df50313e7d None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:28.832087 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:28.847029 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa50dc1-013b-4144-8b9e-f97d7fc73718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:28.847686 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa50dc1-013b-4144-8b9e-f97d7fc73718 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:28.847969 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa50dc1-013b-4144-8b9e-f97d7fc73718 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:28.848248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa50dc1-013b-4144-8b9e-f97d7fc73718 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:28.849607 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be6595c-23ef-473e-a0b4-f02cd975dd0c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:28.866498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa50dc1-013b-4144-8b9e-f97d7fc73718 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:28.872076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-afa50dc1-013b-4144-8b9e-f97d7fc73718 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:28.872076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:28.872076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:28.872076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:28.872076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:28.872076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:28.872076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:28.872076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:28.872076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:28.872076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:28.872076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:28.872076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:28.872076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:28.872076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:28.872076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:28.872076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:28.873107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:28.873107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:28.873107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:28.873107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:28.873107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:28.873107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:28.873107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:28.873107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:28.873107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:28.873107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:28.873107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:28.873107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:28.873107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:28.873107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:28.873107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:28.873107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:28.873941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:28.873941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:28.873941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:28.873941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:28.873941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:28.873941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:28.873941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:28.873941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:28.873941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:28.873941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:28.873941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:28.873941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:28.873941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 135/276] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:28.874693 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa50dc1-013b-4144-8b9e-f97d7fc73718 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:28.902131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3730dd-20b9-40cd-b459-89628e2fdd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:28.902784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3730dd-20b9-40cd-b459-89628e2fdd51 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:28.903066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3730dd-20b9-40cd-b459-89628e2fdd51 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:28.903341 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3730dd-20b9-40cd-b459-89628e2fdd51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:28.904483 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3730dd-20b9-40cd-b459-89628e2fdd51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:28.905983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3730dd-20b9-40cd-b459-89628e2fdd51 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:28.907764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 142/277] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:46:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:28.907872 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:28.912896 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3730dd-20b9-40cd-b459-89628e2fdd51 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:28.984639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa50dc1-013b-4144-8b9e-f97d7fc73718 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:29.018912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4dc21-9100-4d54-94a7-e5b29b746d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:29.019554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4dc21-9100-4d54-94a7-e5b29b746d7e None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:29.019843 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4dc21-9100-4d54-94a7-e5b29b746d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:29.020168 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4dc21-9100-4d54-94a7-e5b29b746d7e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:29.031008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4dc21-9100-4d54-94a7-e5b29b746d7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:29.031158 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3730dd-20b9-40cd-b459-89628e2fdd51 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:29.035971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-7ef4dc21-9100-4d54-94a7-e5b29b746d7e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:46:29.035971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:29.035971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:29.035971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:29.035971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:29.035971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:29.035971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:29.035971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:29.035971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:29.035971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:29.035971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:29.035971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:29.035971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:29.035971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:29.035971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:29.035971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:29.036900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:29.036900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:29.036900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:29.036900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:29.036900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:29.036900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:29.036900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:29.036900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:29.036900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:29.036900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:29.036900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:29.036900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:29.036900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:29.036900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:29.036900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:29.036900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:29.037726 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:29.037726 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:29.037726 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:29.037726 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:29.037726 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:29.037726 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:29.037726 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:29.037726 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:29.037726 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:29.037726 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:29.037726 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:46:29.037726 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:29.042178 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 136/278] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:29.044078 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4dc21-9100-4d54-94a7-e5b29b746d7e None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:29.075484 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d090025-76c5-478d-bb18-7870e9164a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:29.075484 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d090025-76c5-478d-bb18-7870e9164a93 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:29.075484 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d090025-76c5-478d-bb18-7870e9164a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:29.075484 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d090025-76c5-478d-bb18-7870e9164a93 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:29.075914 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d090025-76c5-478d-bb18-7870e9164a93 None admin] RBAC: Adding query filter params (name=member) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:29.075914 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d090025-76c5-478d-bb18-7870e9164a93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:29.075914 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d090025-76c5-478d-bb18-7870e9164a93 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:29.083892 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 143/279] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:29.084816 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d090025-76c5-478d-bb18-7870e9164a93 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:29.106819 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:29.151133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:29.162766 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4dc21-9100-4d54-94a7-e5b29b746d7e None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:29.199202 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d090025-76c5-478d-bb18-7870e9164a93 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:29.200212 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb5267-2db2-4202-ac5e-102cabc4a5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:29.200941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb5267-2db2-4202-ac5e-102cabc4a5d3 None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:29.201323 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb5267-2db2-4202-ac5e-102cabc4a5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:29.201708 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb5267-2db2-4202-ac5e-102cabc4a5d3 None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:29.202523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb5267-2db2-4202-ac5e-102cabc4a5d3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:29.204072 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb5267-2db2-4202-ac5e-102cabc4a5d3 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:29.234248 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3ef97-1e9d-45bd-97b9-7d691efd304b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:29.234863 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3ef97-1e9d-45bd-97b9-7d691efd304b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:29.235209 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3ef97-1e9d-45bd-97b9-7d691efd304b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:29.235486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a3ef97-1e9d-45bd-97b9-7d691efd304b None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:29.236197 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3ef97-1e9d-45bd-97b9-7d691efd304b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:29.236506 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3ef97-1e9d-45bd-97b9-7d691efd304b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:29.237875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a3ef97-1e9d-45bd-97b9-7d691efd304b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:29.245696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 144/280] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:29.246549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3ef97-1e9d-45bd-97b9-7d691efd304b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:29.294494 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 137/281] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 2 08:46:29 2022] POST /identity/v3/users => generated 293 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:46:29.295208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb5267-2db2-4202-ac5e-102cabc4a5d3 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:29.358187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:29.392050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a3ef97-1e9d-45bd-97b9-7d691efd304b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:29.417823 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb5267-2db2-4202-ac5e-102cabc4a5d3 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:29.443523 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f3233a-8155-42b0-bf09-a58d50466a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:29.445826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3233a-8155-42b0-bf09-a58d50466a1f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:29.446324 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3233a-8155-42b0-bf09-a58d50466a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:29.447029 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f3233a-8155-42b0-bf09-a58d50466a1f None admin] PATH_INFO: `/v3/users/e8888388af7c458db9680b5559bb1c3d` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:29.458662 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3e1ca-fd80-4599-b577-0a6c76cd4080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:29.460488 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3e1ca-fd80-4599-b577-0a6c76cd4080 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:29.461487 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3e1ca-fd80-4599-b577-0a6c76cd4080 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:29.461667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3e1ca-fd80-4599-b577-0a6c76cd4080 None admin] PATH_INFO: `/v3/users/8db4498da11d43959b4197493394e55a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:29.462948 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c3e1ca-fd80-4599-b577-0a6c76cd4080 None admin] RBAC: Authorizing `identity:get_user(user_id=8db4498da11d43959b4197493394e55a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:29.464519 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c3e1ca-fd80-4599-b577-0a6c76cd4080 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:29.470044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 138/282] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:29 2022] GET /identity/v3/users/8db4498da11d43959b4197493394e55a => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:29.470196 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f3233a-8155-42b0-bf09-a58d50466a1f None admin] RBAC: Authorizing `identity:get_user(user_id=e8888388af7c458db9680b5559bb1c3d)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:29.470196 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f3233a-8155-42b0-bf09-a58d50466a1f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:29.470754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 145/283] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:29 2022] GET /identity/v3/users/e8888388af7c458db9680b5559bb1c3d => generated 293 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:29.472370 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3e1ca-fd80-4599-b577-0a6c76cd4080 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:29.477571 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f3233a-8155-42b0-bf09-a58d50466a1f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:29.497915 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:29.612843 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f3233a-8155-42b0-bf09-a58d50466a1f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:29.613528 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3e1ca-fd80-4599-b577-0a6c76cd4080 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:29.653594 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a34f85-9b9e-4c78-b940-595644c2ee23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:29.654267 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a34f85-9b9e-4c78-b940-595644c2ee23 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:29.654612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a34f85-9b9e-4c78-b940-595644c2ee23 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:29.654810 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a34f85-9b9e-4c78-b940-595644c2ee23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:29.655616 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a34f85-9b9e-4c78-b940-595644c2ee23 None admin] RBAC: Adding query filter params (role.id=68fe920c9be84b3aa1d35bdce554d96a, user.id=e8888388af7c458db9680b5559bb1c3d, scope.system=all) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:29.655931 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a34f85-9b9e-4c78-b940-595644c2ee23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:29.657379 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a34f85-9b9e-4c78-b940-595644c2ee23 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:29.661412 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:29.666509 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 146/284] 10.222.0.46 () {56 vars in 1308 bytes} [Tue Aug 2 08:46:29 2022] GET /identity/v3/role_assignments?user.id=e8888388af7c458db9680b5559bb1c3d&scope.system=all&role.id=68fe920c9be84b3aa1d35bdce554d96a => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:29.667864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65009a65-045f-41b3-83d8-4b5cefa034b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:29.668523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65009a65-045f-41b3-83d8-4b5cefa034b1 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:29.668899 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65009a65-045f-41b3-83d8-4b5cefa034b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:29.669089 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65009a65-045f-41b3-83d8-4b5cefa034b1 None admin] PATH_INFO: `/v3/projects/f8bf83f1361247688208d846845c9d82` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:29.673938 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65009a65-045f-41b3-83d8-4b5cefa034b1 None admin] RBAC: Authorizing `identity:get_project(project_id=f8bf83f1361247688208d846845c9d82)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:29.673938 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65009a65-045f-41b3-83d8-4b5cefa034b1 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:29.674270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 139/285] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:29 2022] GET /identity/v3/projects/f8bf83f1361247688208d846845c9d82 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:29.686074 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a34f85-9b9e-4c78-b940-595644c2ee23 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:29.796547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a34f85-9b9e-4c78-b940-595644c2ee23 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:29.841520 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af27a2b8-c74c-4481-b1d4-2f1dbaaf1298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:29.842203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27a2b8-c74c-4481-b1d4-2f1dbaaf1298 None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:29.842482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27a2b8-c74c-4481-b1d4-2f1dbaaf1298 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:29.842756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27a2b8-c74c-4481-b1d4-2f1dbaaf1298 None admin] PATH_INFO: `/v3/projects/f8bf83f1361247688208d846845c9d82/users/8db4498da11d43959b4197493394e55a/roles/3c439be3950947478195dcc08835a9b5` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:29.844761 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af27a2b8-c74c-4481-b1d4-2f1dbaaf1298 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8bf83f1361247688208d846845c9d82, user_id=8db4498da11d43959b4197493394e55a, role_id=3c439be3950947478195dcc08835a9b5)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:29.846355 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af27a2b8-c74c-4481-b1d4-2f1dbaaf1298 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:29.974073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 147/286] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:46:29 2022] PUT /identity/v3/projects/f8bf83f1361247688208d846845c9d82/users/8db4498da11d43959b4197493394e55a/roles/3c439be3950947478195dcc08835a9b5 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:46:30.130499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63897ed0-78df-494e-9a1d-44333cc230b3 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:30.130791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63897ed0-78df-494e-9a1d-44333cc230b3 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:30.131082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63897ed0-78df-494e-9a1d-44333cc230b3 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:30.133422 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 140/287] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:30 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:30.139327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef50df1-6cd9-4d21-8b16-585fcd2260c4 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:30.139614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef50df1-6cd9-4d21-8b16-585fcd2260c4 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:30.139893 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef50df1-6cd9-4d21-8b16-585fcd2260c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:30.162634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-8ef50df1-6cd9-4d21-8b16-585fcd2260c4 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:30.164001 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8ef50df1-6cd9-4d21-8b16-585fcd2260c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:30.239766 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:30.251850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 148/288] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:30.280143 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63897ed0-78df-494e-9a1d-44333cc230b3 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:30.389616 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:30.419021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63897ed0-78df-494e-9a1d-44333cc230b3 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:30.464311 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbf1f8-75b2-4acb-a714-615b06d80e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:30.464967 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbf1f8-75b2-4acb-a714-615b06d80e06 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:30.465253 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbf1f8-75b2-4acb-a714-615b06d80e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:30.465534 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbf1f8-75b2-4acb-a714-615b06d80e06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:30.482085 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbf1f8-75b2-4acb-a714-615b06d80e06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:30.487328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-0adbf1f8-75b2-4acb-a714-615b06d80e06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:30.487328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:30.487328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:30.487328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:30.487328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:30.487328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:30.487328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:30.487328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:30.487328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:30.487328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:30.487328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:30.487328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:30.487328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:30.487328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:30.487328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:30.487328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:30.488208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:30.488208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:30.488208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:30.488208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:30.488208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:30.488208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:30.488208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:30.488208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:30.488208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:30.488208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:30.488208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:30.488208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:30.488208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:30.488208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:30.488208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:30.488208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:30.489007 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:30.489007 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:30.489007 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:30.489007 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:30.489007 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:30.489007 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:30.489007 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:30.489007 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:30.489007 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:30.489007 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:30.489007 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:30.489007 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:30.489007 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 141/289] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:30.499076 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef50df1-6cd9-4d21-8b16-585fcd2260c4 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:30.649189 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef50df1-6cd9-4d21-8b16-585fcd2260c4 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:30.696332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb3f180-4ec0-4aeb-8347-3ab3e14e9fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:30.696961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3f180-4ec0-4aeb-8347-3ab3e14e9fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:30.697246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3f180-4ec0-4aeb-8347-3ab3e14e9fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:30.697783 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3f180-4ec0-4aeb-8347-3ab3e14e9fe9 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:30.698553 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb3f180-4ec0-4aeb-8347-3ab3e14e9fe9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:30.698860 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb3f180-4ec0-4aeb-8347-3ab3e14e9fe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:30.700230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb3f180-4ec0-4aeb-8347-3ab3e14e9fe9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:30.708951 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 149/290] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:30.714721 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbf1f8-75b2-4acb-a714-615b06d80e06 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:30.854715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbf1f8-75b2-4acb-a714-615b06d80e06 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:30.924050 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7cb92-b7b3-46bf-b534-f159c7c2b3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:30.925318 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7cb92-b7b3-46bf-b534-f159c7c2b3cb None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:30.925581 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7cb92-b7b3-46bf-b534-f159c7c2b3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:30.926650 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7cb92-b7b3-46bf-b534-f159c7c2b3cb None admin] PATH_INFO: `/v3/users/1394e6b314554594bf617f454f136113` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:30.928234 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c7cb92-b7b3-46bf-b534-f159c7c2b3cb None admin] RBAC: Authorizing `identity:get_user(user_id=1394e6b314554594bf617f454f136113)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:30.933530 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c7cb92-b7b3-46bf-b534-f159c7c2b3cb None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:30.933530 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 142/291] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:30 2022] GET /identity/v3/users/1394e6b314554594bf617f454f136113 => generated 297 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:30.947128 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb3f180-4ec0-4aeb-8347-3ab3e14e9fe9 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:31.032134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:31.097506 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb3f180-4ec0-4aeb-8347-3ab3e14e9fe9 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:31.139228 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdba59-97ff-447d-a477-6f8e1d5dcea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:31.139781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdba59-97ff-447d-a477-6f8e1d5dcea7 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:31.140065 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdba59-97ff-447d-a477-6f8e1d5dcea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:31.140339 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdba59-97ff-447d-a477-6f8e1d5dcea7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:31.236488 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fdba59-97ff-447d-a477-6f8e1d5dcea7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:31.237951 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fdba59-97ff-447d-a477-6f8e1d5dcea7 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:31.302292 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:31.363332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 150/292] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:46:31.371132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7cb92-b7b3-46bf-b534-f159c7c2b3cb None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:31.387284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:31.502984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7cb92-b7b3-46bf-b534-f159c7c2b3cb None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:31.545791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2cc060-c85b-4bcd-93bc-5638a9ec1b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:31.546464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cc060-c85b-4bcd-93bc-5638a9ec1b45 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:31.546749 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cc060-c85b-4bcd-93bc-5638a9ec1b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:31.547026 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cc060-c85b-4bcd-93bc-5638a9ec1b45 None admin] PATH_INFO: `/v3/projects/f8bf83f1361247688208d846845c9d82` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:31.550594 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2cc060-c85b-4bcd-93bc-5638a9ec1b45 None admin] RBAC: Authorizing `identity:get_project(project_id=f8bf83f1361247688208d846845c9d82)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:31.552039 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2cc060-c85b-4bcd-93bc-5638a9ec1b45 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:31.553735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 143/293] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:31 2022] GET /identity/v3/projects/f8bf83f1361247688208d846845c9d82 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:31.560470 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdba59-97ff-447d-a477-6f8e1d5dcea7 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:31.716048 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdba59-97ff-447d-a477-6f8e1d5dcea7 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:31.759365 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5224e3-60a4-4ea9-b27a-295e538bc8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:31.759911 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5224e3-60a4-4ea9-b27a-295e538bc8cb None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:31.760245 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5224e3-60a4-4ea9-b27a-295e538bc8cb None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:31.760527 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5224e3-60a4-4ea9-b27a-295e538bc8cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:31.761326 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5224e3-60a4-4ea9-b27a-295e538bc8cb None admin] RBAC: Adding query filter params (role.id=68fe920c9be84b3aa1d35bdce554d96a, scope.project.id=f8bf83f1361247688208d846845c9d82, user.id=1394e6b314554594bf617f454f136113) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:31.761637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5224e3-60a4-4ea9-b27a-295e538bc8cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:31.763104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5224e3-60a4-4ea9-b27a-295e538bc8cb None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:31.774297 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 151/294] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:46:31 2022] GET /identity/v3/role_assignments?user.id=1394e6b314554594bf617f454f136113&scope.project.id=f8bf83f1361247688208d846845c9d82&role.id=68fe920c9be84b3aa1d35bdce554d96a => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:32.009558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3834655-a791-485b-9c31-5eeed5d17182 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:32.010230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3834655-a791-485b-9c31-5eeed5d17182 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:32.010733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3834655-a791-485b-9c31-5eeed5d17182 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:32.012599 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 144/295] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:32.018323 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4d51f-106d-45b1-a053-f0d48e8a66f5 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:32.018911 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4d51f-106d-45b1-a053-f0d48e8a66f5 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:32.019390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a4d51f-106d-45b1-a053-f0d48e8a66f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:32.038825 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-20a4d51f-106d-45b1-a053-f0d48e8a66f5 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:32.040368 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-20a4d51f-106d-45b1-a053-f0d48e8a66f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:32.053170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:32.117995 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 152/296] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:32.138389 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3834655-a791-485b-9c31-5eeed5d17182 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:32.160184 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1816e4-9a9f-42ad-af2a-80604c8ae074 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:32.160497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1816e4-9a9f-42ad-af2a-80604c8ae074 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:32.160750 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1816e4-9a9f-42ad-af2a-80604c8ae074 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:32.162388 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 153/297] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:46:32.170660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a80a9-acc1-42c7-a268-0183a8c8173d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:32.170953 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a80a9-acc1-42c7-a268-0183a8c8173d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:32.171232 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a80a9-acc1-42c7-a268-0183a8c8173d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:32.192173 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-000a80a9-acc1-42c7-a268-0183a8c8173d None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:32.193773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-000a80a9-acc1-42c7-a268-0183a8c8173d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:32.227021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:32.257769 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3834655-a791-485b-9c31-5eeed5d17182 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:32.283886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 154/298] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:46:32.285109 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9173773-c7be-43f1-bcff-c7054effee6b None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:32.285517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9173773-c7be-43f1-bcff-c7054effee6b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:32.285800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9173773-c7be-43f1-bcff-c7054effee6b None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:32.287329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 155/299] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:32.292919 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d78b2-de1d-4cee-a12f-3c36d932eb42 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:32.293256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d78b2-de1d-4cee-a12f-3c36d932eb42 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:32.293546 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d78b2-de1d-4cee-a12f-3c36d932eb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:32.297401 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099363e-5194-4370-9ac7-46979a4c0f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:32.298336 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099363e-5194-4370-9ac7-46979a4c0f48 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:32.298823 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099363e-5194-4370-9ac7-46979a4c0f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:32.299298 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099363e-5194-4370-9ac7-46979a4c0f48 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:32.310597 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099363e-5194-4370-9ac7-46979a4c0f48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:32.312044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-631d78b2-de1d-4cee-a12f-3c36d932eb42 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:32.313247 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-631d78b2-de1d-4cee-a12f-3c36d932eb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:32.316657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-2099363e-5194-4370-9ac7-46979a4c0f48 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:46:32.316657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:32.316657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:32.316657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:32.316657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:32.316657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:32.316657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:32.316657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:32.316657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:32.316657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:32.316657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:32.316657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:32.316657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:32.316657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:32.316657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:32.316657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:32.317942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:32.317942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:32.317942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:32.317942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:32.317942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:32.317942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:32.317942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:32.317942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:32.317942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:32.317942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:32.317942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:32.317942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:32.317942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:32.317942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:32.317942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:32.317942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:32.318748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:32.318748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:32.318748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:32.318748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:32.318748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:32.318748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:32.318748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:32.318748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:32.318748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:32.318748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:32.318748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:46:32.318748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:32.321507 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 145/300] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:46:32.327395 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099363e-5194-4370-9ac7-46979a4c0f48 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:32.383945 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 156/301] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:46:32.384697 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d78b2-de1d-4cee-a12f-3c36d932eb42 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:32.405732 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:32.447853 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099363e-5194-4370-9ac7-46979a4c0f48 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:32.486958 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae455b0-20b8-4cad-9924-d7a03806f19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:32.487730 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae455b0-20b8-4cad-9924-d7a03806f19d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:32.488303 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae455b0-20b8-4cad-9924-d7a03806f19d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:32.488609 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae455b0-20b8-4cad-9924-d7a03806f19d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:32.489752 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae455b0-20b8-4cad-9924-d7a03806f19d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:32.492260 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae455b0-20b8-4cad-9924-d7a03806f19d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:32.498856 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 146/302] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:46:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:32.498856 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae455b0-20b8-4cad-9924-d7a03806f19d None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:32.503323 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d78b2-de1d-4cee-a12f-3c36d932eb42 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:32.504000 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:32.540524 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165908a6-7052-43ad-8849-64aa68e13527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:32.541071 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165908a6-7052-43ad-8849-64aa68e13527 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:32.541354 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165908a6-7052-43ad-8849-64aa68e13527 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:32.541631 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165908a6-7052-43ad-8849-64aa68e13527 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:32.542383 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165908a6-7052-43ad-8849-64aa68e13527 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:32.542692 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165908a6-7052-43ad-8849-64aa68e13527 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:32.544067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165908a6-7052-43ad-8849-64aa68e13527 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:32.552443 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 157/303] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:32.553212 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165908a6-7052-43ad-8849-64aa68e13527 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:32.615690 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae455b0-20b8-4cad-9924-d7a03806f19d None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:32.624305 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:32.650524 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0f688-2d1f-4215-8e45-2bd5ccca9e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:32.651159 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0f688-2d1f-4215-8e45-2bd5ccca9e1c None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:32.651436 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0f688-2d1f-4215-8e45-2bd5ccca9e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:32.651826 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0f688-2d1f-4215-8e45-2bd5ccca9e1c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:32.658460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165908a6-7052-43ad-8849-64aa68e13527 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:32.662288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa0f688-2d1f-4215-8e45-2bd5ccca9e1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:32.668248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-caa0f688-2d1f-4215-8e45-2bd5ccca9e1c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:32.668248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:32.668248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:32.668248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:32.668248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:32.668248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:32.668248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:32.668248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:32.668248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:32.668248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:32.668248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:32.668248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:32.668248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:32.668248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:32.668248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:32.668248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:32.669132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:32.669132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:32.669132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:32.669132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:32.669132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:32.669132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:32.669132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:32.669132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:32.669132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:32.669132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:32.669132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:32.669132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:32.669132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:32.669132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:32.669132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:32.669132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:32.669956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:32.669956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:32.669956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:32.669956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:32.669956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:32.669956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:32.669956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:32.669956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:32.669956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:32.669956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:32.669956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:32.669956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:32.669956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 147/304] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:32.670730 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0f688-2d1f-4215-8e45-2bd5ccca9e1c None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:32.694064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2c0412-0d86-419a-91a7-dd53754ffd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:32.695517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c0412-0d86-419a-91a7-dd53754ffd4a None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:32.695806 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c0412-0d86-419a-91a7-dd53754ffd4a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:32.696083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2c0412-0d86-419a-91a7-dd53754ffd4a None admin] PATH_INFO: `/v3/groups` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:32.696736 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c0412-0d86-419a-91a7-dd53754ffd4a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:32.698474 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2c0412-0d86-419a-91a7-dd53754ffd4a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:32.734088 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:32.769842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0f688-2d1f-4215-8e45-2bd5ccca9e1c None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:32.775201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 158/305] 10.222.0.46 () {60 vars in 1134 bytes} [Tue Aug 2 08:46:32 2022] POST /identity/v3/groups => generated 229 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:46:32.776471 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8c335-364d-4912-a421-3396b8db6d81 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:32.776766 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8c335-364d-4912-a421-3396b8db6d81 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:32.777047 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8c335-364d-4912-a421-3396b8db6d81 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:32.778357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 159/306] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:32.779110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8c335-364d-4912-a421-3396b8db6d81 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:32.825828 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85f5d9-4f65-464f-87da-331ea18a03e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:32.825828 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85f5d9-4f65-464f-87da-331ea18a03e8 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:32.825828 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85f5d9-4f65-464f-87da-331ea18a03e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:32.825828 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85f5d9-4f65-464f-87da-331ea18a03e8 None admin] PATH_INFO: `/v3/users/85dd4c5d75954e7abdfa21159112b03d` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:32.841804 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85f5d9-4f65-464f-87da-331ea18a03e8 None admin] RBAC: Authorizing `identity:get_user(user_id=85dd4c5d75954e7abdfa21159112b03d)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:32.843670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85f5d9-4f65-464f-87da-331ea18a03e8 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:32.846044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 148/307] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:32 2022] GET /identity/v3/users/85dd4c5d75954e7abdfa21159112b03d => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:32.847561 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4aec8-475f-4056-afe6-d467bc2d0a2d None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:32.847843 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4aec8-475f-4056-afe6-d467bc2d0a2d None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:32.848113 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d4aec8-475f-4056-afe6-d467bc2d0a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:32.868693 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-c9d4aec8-475f-4056-afe6-d467bc2d0a2d None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:32.869931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-c9d4aec8-475f-4056-afe6-d467bc2d0a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:32.904205 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8c335-364d-4912-a421-3396b8db6d81 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:32.933109 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:32.943447 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdca32b-ddd4-4857-845d-0b79f52473b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:32.944227 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdca32b-ddd4-4857-845d-0b79f52473b1 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:32.944847 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdca32b-ddd4-4857-845d-0b79f52473b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:32.945131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdca32b-ddd4-4857-845d-0b79f52473b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:32.946213 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdca32b-ddd4-4857-845d-0b79f52473b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:32.946519 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdca32b-ddd4-4857-845d-0b79f52473b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:32.948146 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdca32b-ddd4-4857-845d-0b79f52473b1 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:32.952941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 149/308] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:32.954217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4aec8-475f-4056-afe6-d467bc2d0a2d None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:32.957156 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 160/309] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:32.963960 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdca32b-ddd4-4857-845d-0b79f52473b1 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:33.013255 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:33.061644 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d4aec8-475f-4056-afe6-d467bc2d0a2d None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:33.081552 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdca32b-ddd4-4857-845d-0b79f52473b1 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:33.097143 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce201767-6968-44f7-b90c-b6f3835246ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:33.098002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce201767-6968-44f7-b90c-b6f3835246ba None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:33.098656 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce201767-6968-44f7-b90c-b6f3835246ba None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:33.099081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce201767-6968-44f7-b90c-b6f3835246ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:33.100090 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce201767-6968-44f7-b90c-b6f3835246ba None admin] RBAC: Adding query filter params (role.id=3c439be3950947478195dcc08835a9b5, user.id=85dd4c5d75954e7abdfa21159112b03d, scope.system=all) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:33.100561 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce201767-6968-44f7-b90c-b6f3835246ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:33.102819 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce201767-6968-44f7-b90c-b6f3835246ba None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:33.117483 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3bb238-cb30-4547-8dd5-00d8bdc24988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:33.117914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 150/310] 10.222.0.46 () {56 vars in 1308 bytes} [Tue Aug 2 08:46:32 2022] GET /identity/v3/role_assignments?user.id=85dd4c5d75954e7abdfa21159112b03d&scope.system=all&role.id=3c439be3950947478195dcc08835a9b5 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:33.118271 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3bb238-cb30-4547-8dd5-00d8bdc24988 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:33.118558 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3bb238-cb30-4547-8dd5-00d8bdc24988 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:33.118834 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3bb238-cb30-4547-8dd5-00d8bdc24988 None admin] PATH_INFO: `/v3/users/e8888388af7c458db9680b5559bb1c3d` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:33.119509 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce201767-6968-44f7-b90c-b6f3835246ba None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:33.120049 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3bb238-cb30-4547-8dd5-00d8bdc24988 None admin] RBAC: Authorizing `identity:get_user(user_id=e8888388af7c458db9680b5559bb1c3d)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:33.121520 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3bb238-cb30-4547-8dd5-00d8bdc24988 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:33.123679 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 161/311] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:32 2022] GET /identity/v3/users/e8888388af7c458db9680b5559bb1c3d => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:33.130220 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3bb238-cb30-4547-8dd5-00d8bdc24988 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:33.209129 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:33.261746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3bb238-cb30-4547-8dd5-00d8bdc24988 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:33.272669 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce201767-6968-44f7-b90c-b6f3835246ba None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:33.295969 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:33.310114 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e00628-0370-4345-8d23-06de9560cbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:33.311006 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e00628-0370-4345-8d23-06de9560cbac None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:33.311441 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e00628-0370-4345-8d23-06de9560cbac None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:33.311867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e00628-0370-4345-8d23-06de9560cbac None admin] PATH_INFO: `/v3/system/users/e8888388af7c458db9680b5559bb1c3d/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:33.313546 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e00628-0370-4345-8d23-06de9560cbac None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e8888388af7c458db9680b5559bb1c3d, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:33.314507 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c5377-60c2-4ade-9e97-5d8ce408c1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:33.315154 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c5377-60c2-4ade-9e97-5d8ce408c1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:33.315437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c5377-60c2-4ade-9e97-5d8ce408c1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:33.315609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e00628-0370-4345-8d23-06de9560cbac None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:33.315711 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c5377-60c2-4ade-9e97-5d8ce408c1e8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:33.327917 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405c5377-60c2-4ade-9e97-5d8ce408c1e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:33.333876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-405c5377-60c2-4ade-9e97-5d8ce408c1e8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:46:33.333876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:33.333876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:33.333876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:33.333876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:33.333876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:33.333876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:33.333876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:33.333876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:33.333876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:33.333876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:33.333876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:33.333876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:33.333876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:33.333876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:33.333876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:33.334796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:33.334796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:33.334796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:33.334796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:33.334796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:33.334796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:33.334796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:33.334796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:33.334796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:33.334796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:33.334796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:33.334796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:33.334796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:33.334796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:33.334796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:33.334796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:33.335735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:33.335735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:33.335735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:33.335735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:33.335735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:33.335735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:33.335735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:33.335735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:33.335735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:33.335735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:33.335735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:46:33.335735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:33.335735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 151/312] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:33.340927 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c5377-60c2-4ade-9e97-5d8ce408c1e8 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:33.405170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 162/313] 10.222.0.46 () {58 vars in 1334 bytes} [Tue Aug 2 08:46:33 2022] PUT /identity/v3/system/users/e8888388af7c458db9680b5559bb1c3d/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 08:46:33.463722 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c5377-60c2-4ade-9e97-5d8ce408c1e8 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:33.504794 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d56e6-a3c2-4a13-a0a1-e04e79987cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:33.506295 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d56e6-a3c2-4a13-a0a1-e04e79987cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:33.506788 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d56e6-a3c2-4a13-a0a1-e04e79987cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:33.507845 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d56e6-a3c2-4a13-a0a1-e04e79987cc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:33.508799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d56e6-a3c2-4a13-a0a1-e04e79987cc7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:33.509313 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d56e6-a3c2-4a13-a0a1-e04e79987cc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:33.510984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d56e6-a3c2-4a13-a0a1-e04e79987cc7 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:33.521479 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 152/314] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:33.528166 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e00628-0370-4345-8d23-06de9560cbac None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:33.640493 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e00628-0370-4345-8d23-06de9560cbac None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:33.665074 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:33.676886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b8eb8-6e4c-4021-ae3f-a779e26494bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:33.678837 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b8eb8-6e4c-4021-ae3f-a779e26494bd None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:33.678837 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b8eb8-6e4c-4021-ae3f-a779e26494bd None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:33.678837 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8b8eb8-6e4c-4021-ae3f-a779e26494bd None admin] PATH_INFO: `/v3/users/8db4498da11d43959b4197493394e55a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:33.680248 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b8eb8-6e4c-4021-ae3f-a779e26494bd None admin] RBAC: Authorizing `identity:get_user(user_id=8db4498da11d43959b4197493394e55a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:33.681910 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8b8eb8-6e4c-4021-ae3f-a779e26494bd None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:33.684046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 163/315] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:33 2022] GET /identity/v3/users/8db4498da11d43959b4197493394e55a => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:33.689444 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d56e6-a3c2-4a13-a0a1-e04e79987cc7 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:33.733699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:33.798885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d56e6-a3c2-4a13-a0a1-e04e79987cc7 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:33.837120 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec377b9-07a4-474b-aa30-94e630a90200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:33.838127 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec377b9-07a4-474b-aa30-94e630a90200 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:33.838655 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec377b9-07a4-474b-aa30-94e630a90200 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:33.839135 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec377b9-07a4-474b-aa30-94e630a90200 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:33.912383 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec377b9-07a4-474b-aa30-94e630a90200 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:33.914909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec377b9-07a4-474b-aa30-94e630a90200 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:33.958527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:34.024742 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 153/316] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:46:34.029742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b8eb8-6e4c-4021-ae3f-a779e26494bd None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:34.136544 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8b8eb8-6e4c-4021-ae3f-a779e26494bd None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:34.172338 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a1947a-ae72-4d93-a7cb-da81a85fe336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:34.172912 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1947a-ae72-4d93-a7cb-da81a85fe336 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:34.173197 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1947a-ae72-4d93-a7cb-da81a85fe336 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:34.173476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a1947a-ae72-4d93-a7cb-da81a85fe336 None admin] PATH_INFO: `/v3/projects/f8bf83f1361247688208d846845c9d82` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:34.174689 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a1947a-ae72-4d93-a7cb-da81a85fe336 None admin] RBAC: Authorizing `identity:get_project(project_id=f8bf83f1361247688208d846845c9d82)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:34.176128 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a1947a-ae72-4d93-a7cb-da81a85fe336 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:34.181985 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 164/317] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:34 2022] GET /identity/v3/projects/f8bf83f1361247688208d846845c9d82 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:34.184741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec377b9-07a4-474b-aa30-94e630a90200 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:34.295231 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec377b9-07a4-474b-aa30-94e630a90200 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:34.330367 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1734e5-3973-4876-85d0-ebf769be36a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:34.331361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1734e5-3973-4876-85d0-ebf769be36a3 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:34.331838 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1734e5-3973-4876-85d0-ebf769be36a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:34.332304 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1734e5-3973-4876-85d0-ebf769be36a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:34.333291 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1734e5-3973-4876-85d0-ebf769be36a3 None admin] RBAC: Adding query filter params (role.id=3c439be3950947478195dcc08835a9b5, scope.project.id=f8bf83f1361247688208d846845c9d82, user.id=8db4498da11d43959b4197493394e55a) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:34.333794 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1734e5-3973-4876-85d0-ebf769be36a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:34.336151 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1734e5-3973-4876-85d0-ebf769be36a3 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:34.342274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 154/318] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:46:34 2022] GET /identity/v3/role_assignments?user.id=8db4498da11d43959b4197493394e55a&scope.project.id=f8bf83f1361247688208d846845c9d82&role.id=3c439be3950947478195dcc08835a9b5 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:34.932052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592705b9-d452-4111-a970-24753b34d408 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:34.932446 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592705b9-d452-4111-a970-24753b34d408 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:34.932623 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592705b9-d452-4111-a970-24753b34d408 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:34.936588 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 165/319] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:34.940852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ea255-d457-462c-b937-88b7305dc273 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:34.941272 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ea255-d457-462c-b937-88b7305dc273 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:34.941410 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ea255-d457-462c-b937-88b7305dc273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:34.949015 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:34.962646 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-3a2ea255-d457-462c-b937-88b7305dc273 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:34.963850 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-3a2ea255-d457-462c-b937-88b7305dc273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:35.033481 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 155/320] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:35.055989 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592705b9-d452-4111-a970-24753b34d408 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:35.123052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:35.170951 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592705b9-d452-4111-a970-24753b34d408 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:35.190512 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5b8e1-10d9-4af7-af3f-fb8480a45abd None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:35.190843 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5b8e1-10d9-4af7-af3f-fb8480a45abd None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:35.191024 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5b8e1-10d9-4af7-af3f-fb8480a45abd None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:35.192493 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 156/321] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:46:35.198158 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06629d3a-271f-433e-94fb-955d6119ccfd None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:35.198680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06629d3a-271f-433e-94fb-955d6119ccfd None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:35.199200 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06629d3a-271f-433e-94fb-955d6119ccfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:35.214142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efabcd-8486-437a-a864-aecb981fb162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:35.214867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efabcd-8486-437a-a864-aecb981fb162 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:35.215078 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efabcd-8486-437a-a864-aecb981fb162 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:35.215357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efabcd-8486-437a-a864-aecb981fb162 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:35.216506 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43efabcd-8486-437a-a864-aecb981fb162 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:35.218022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43efabcd-8486-437a-a864-aecb981fb162 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:35.219887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 166/322] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:46:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:35.220068 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-06629d3a-271f-433e-94fb-955d6119ccfd None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:35.221297 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-06629d3a-271f-433e-94fb-955d6119ccfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:35.225249 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efabcd-8486-437a-a864-aecb981fb162 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:35.297526 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 157/323] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:46:35.321004 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06629d3a-271f-433e-94fb-955d6119ccfd None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:35.341659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efabcd-8486-437a-a864-aecb981fb162 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:35.372648 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:35.382140 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e633b0f-e1a0-45cc-9c35-ab9e31b2a0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:35.383506 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e633b0f-e1a0-45cc-9c35-ab9e31b2a0eb None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:35.383789 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e633b0f-e1a0-45cc-9c35-ab9e31b2a0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:35.384099 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e633b0f-e1a0-45cc-9c35-ab9e31b2a0eb None admin] PATH_INFO: `/v3/groups` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:35.384789 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e633b0f-e1a0-45cc-9c35-ab9e31b2a0eb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:35.386274 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e633b0f-e1a0-45cc-9c35-ab9e31b2a0eb None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:35.388740 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:35.428359 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06629d3a-271f-433e-94fb-955d6119ccfd None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:35.465876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d629fdc7-ed7c-463b-84b3-9dc0a6e6a491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:35.466477 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d629fdc7-ed7c-463b-84b3-9dc0a6e6a491 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:35.466762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d629fdc7-ed7c-463b-84b3-9dc0a6e6a491 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:35.467041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d629fdc7-ed7c-463b-84b3-9dc0a6e6a491 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:35.477461 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d629fdc7-ed7c-463b-84b3-9dc0a6e6a491 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:35.482592 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-d629fdc7-ed7c-463b-84b3-9dc0a6e6a491 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:46:35.482592 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:35.482592 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:35.482592 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:35.482592 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:35.482592 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:35.482592 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:35.482592 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:35.482592 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:35.482592 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:35.482592 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:35.482592 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:35.482592 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:35.482592 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:35.482592 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:35.482592 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:35.483842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:35.483842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:35.483842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:35.483842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:35.483842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:35.483842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:35.483842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:35.483842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:35.483842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:35.483842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:35.483842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:35.483842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:35.483842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:35.483842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:35.483842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:35.483842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:35.485002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:35.485002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:35.485002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:35.485002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:35.485002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:35.485002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:35.485002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:35.485002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:35.485002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:35.485002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:35.485002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:46:35.485002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:35.485002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 158/324] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:46:35.490144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d629fdc7-ed7c-463b-84b3-9dc0a6e6a491 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:35.499422 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 167/325] 10.222.0.46 () {60 vars in 1134 bytes} [Tue Aug 2 08:46:35 2022] POST /identity/v3/groups => generated 226 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 08:46:35.537334 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdb4b4-4d9c-41be-8c4d-5b8b13967221 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:35.537628 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdb4b4-4d9c-41be-8c4d-5b8b13967221 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:35.537950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafdb4b4-4d9c-41be-8c4d-5b8b13967221 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:35.539431 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 168/326] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:46:35.544998 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c948b-a937-4328-a812-5631f3513ab9 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:35.545291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c948b-a937-4328-a812-5631f3513ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:35.545579 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c948b-a937-4328-a812-5631f3513ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:35.565116 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-040c948b-a937-4328-a812-5631f3513ab9 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:35.566396 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-040c948b-a937-4328-a812-5631f3513ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:35.608638 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d629fdc7-ed7c-463b-84b3-9dc0a6e6a491 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:35.637606 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:35.648927 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 169/327] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:46:35.650804 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7b356-ff9e-4a92-8145-a2770a3c4ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:35.651533 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7b356-ff9e-4a92-8145-a2770a3c4ddc None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:35.651884 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7b356-ff9e-4a92-8145-a2770a3c4ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:35.652228 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7b356-ff9e-4a92-8145-a2770a3c4ddc None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:35.653020 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7b356-ff9e-4a92-8145-a2770a3c4ddc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:35.653400 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7b356-ff9e-4a92-8145-a2770a3c4ddc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:35.654903 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7b356-ff9e-4a92-8145-a2770a3c4ddc None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:35.664817 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 159/328] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:35.670018 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c948b-a937-4328-a812-5631f3513ab9 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:35.672662 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7b356-ff9e-4a92-8145-a2770a3c4ddc None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:35.798070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7b356-ff9e-4a92-8145-a2770a3c4ddc None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:35.798244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c948b-a937-4328-a812-5631f3513ab9 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:35.821278 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:35.844865 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d0aeb-f3f3-4bb8-b0b7-421b95f57c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:35.845843 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d0aeb-f3f3-4bb8-b0b7-421b95f57c65 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:35.846166 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d0aeb-f3f3-4bb8-b0b7-421b95f57c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:35.846452 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d0aeb-f3f3-4bb8-b0b7-421b95f57c65 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:35.847692 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e1597-bd9f-47ae-ac85-d56f39e578f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:35.848706 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e1597-bd9f-47ae-ac85-d56f39e578f9 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:35.849306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e1597-bd9f-47ae-ac85-d56f39e578f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:35.849644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e1597-bd9f-47ae-ac85-d56f39e578f9 None admin] PATH_INFO: `/v3/users/85dd4c5d75954e7abdfa21159112b03d` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:35.854329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840e1597-bd9f-47ae-ac85-d56f39e578f9 None admin] RBAC: Authorizing `identity:get_user(user_id=85dd4c5d75954e7abdfa21159112b03d)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:35.856399 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840e1597-bd9f-47ae-ac85-d56f39e578f9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:35.858243 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 170/329] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:35 2022] GET /identity/v3/users/85dd4c5d75954e7abdfa21159112b03d => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:35.864066 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d0aeb-f3f3-4bb8-b0b7-421b95f57c65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:35.864448 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e1597-bd9f-47ae-ac85-d56f39e578f9 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:35.870371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-745d0aeb-f3f3-4bb8-b0b7-421b95f57c65 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:35.870371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:35.870371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:35.870371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:35.870371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:35.870371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:35.870371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:35.870371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:35.870371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:35.870371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:35.870371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:35.870371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:35.870371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:35.870371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:35.870371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:35.870371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:35.873731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:35.873731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:35.873731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:35.873731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:35.873731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:35.873731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:35.873731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:35.873731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:35.873731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:35.873731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:35.873731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:35.873731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:35.873731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:35.873731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:35.873731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:35.873731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:35.874911 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:35.874911 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:35.874911 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:35.874911 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:35.874911 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:35.874911 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:35.874911 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:35.874911 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:35.874911 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:35.874911 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:35.874911 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:35.874911 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:35.874911 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 160/330] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:46:35.880003 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d0aeb-f3f3-4bb8-b0b7-421b95f57c65 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:35.935753 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:36.008426 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d0aeb-f3f3-4bb8-b0b7-421b95f57c65 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:36.009468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e1597-bd9f-47ae-ac85-d56f39e578f9 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:36.052801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eabc689-16f3-4460-b669-d9c5b890d35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:36.053598 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eabc689-16f3-4460-b669-d9c5b890d35f None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:36.054032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eabc689-16f3-4460-b669-d9c5b890d35f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:36.054807 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eabc689-16f3-4460-b669-d9c5b890d35f None admin] PATH_INFO: `/v3/system/users/85dd4c5d75954e7abdfa21159112b03d/roles/3c439be3950947478195dcc08835a9b5` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:36.055152 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f6179-dc11-4fd7-81dc-7dac81e0c244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:36.055891 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f6179-dc11-4fd7-81dc-7dac81e0c244 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:36.056184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f6179-dc11-4fd7-81dc-7dac81e0c244 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:36.056384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eabc689-16f3-4460-b669-d9c5b890d35f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=85dd4c5d75954e7abdfa21159112b03d, role_id=3c439be3950947478195dcc08835a9b5)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:36.056801 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f6179-dc11-4fd7-81dc-7dac81e0c244 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:36.057185 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f6179-dc11-4fd7-81dc-7dac81e0c244 None admin] RBAC: Adding query filter params (name=member) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:36.057494 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f6179-dc11-4fd7-81dc-7dac81e0c244 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:36.057785 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eabc689-16f3-4460-b669-d9c5b890d35f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:36.058945 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8f6179-dc11-4fd7-81dc-7dac81e0c244 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:36.068464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 161/331] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:36.074993 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f6179-dc11-4fd7-81dc-7dac81e0c244 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:36.136460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 171/332] 10.222.0.46 () {58 vars in 1334 bytes} [Tue Aug 2 08:46:35 2022] PUT /identity/v3/system/users/85dd4c5d75954e7abdfa21159112b03d/roles/3c439be3950947478195dcc08835a9b5 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:46:36.203922 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f6179-dc11-4fd7-81dc-7dac81e0c244 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:36.213213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:36.245268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20cd87d-627b-4324-b911-f020544968e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:36.246275 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cd87d-627b-4324-b911-f020544968e2 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:36.246747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cd87d-627b-4324-b911-f020544968e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:36.247209 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cd87d-627b-4324-b911-f020544968e2 None admin] PATH_INFO: `/v3/users/e8888388af7c458db9680b5559bb1c3d` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:36.248846 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20cd87d-627b-4324-b911-f020544968e2 None admin] RBAC: Authorizing `identity:get_user(user_id=e8888388af7c458db9680b5559bb1c3d)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:36.250559 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20cd87d-627b-4324-b911-f020544968e2 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:36.252774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 162/333] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:36 2022] GET /identity/v3/users/e8888388af7c458db9680b5559bb1c3d => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:36.259055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eabc689-16f3-4460-b669-d9c5b890d35f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:36.324663 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:36.386750 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eabc689-16f3-4460-b669-d9c5b890d35f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:36.424430 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f95cfe4-193a-4ea3-a980-2d77628d854b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:36.425066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95cfe4-193a-4ea3-a980-2d77628d854b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:36.425334 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95cfe4-193a-4ea3-a980-2d77628d854b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:36.425606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f95cfe4-193a-4ea3-a980-2d77628d854b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:36.426464 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95cfe4-193a-4ea3-a980-2d77628d854b None admin] RBAC: Adding query filter params (role.id=68fe920c9be84b3aa1d35bdce554d96a, user.id=e8888388af7c458db9680b5559bb1c3d, scope.system=all) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:36.426774 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95cfe4-193a-4ea3-a980-2d77628d854b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:36.428208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f95cfe4-193a-4ea3-a980-2d77628d854b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:36.441095 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 172/334] 10.222.0.46 () {56 vars in 1308 bytes} [Tue Aug 2 08:46:36 2022] GET /identity/v3/role_assignments?user.id=e8888388af7c458db9680b5559bb1c3d&scope.system=all&role.id=68fe920c9be84b3aa1d35bdce554d96a => generated 510 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:37.786302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182fe70-fa3a-4dba-afea-09fec41314a9 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:37.787146 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182fe70-fa3a-4dba-afea-09fec41314a9 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:37.787576 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b182fe70-fa3a-4dba-afea-09fec41314a9 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:37.789720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 163/335] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:37.795351 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c9bb6-45e2-46df-8864-0198ddcae33b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:37.811239 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c9bb6-45e2-46df-8864-0198ddcae33b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:37.811239 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0c9bb6-45e2-46df-8864-0198ddcae33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:37.816478 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-2c0c9bb6-45e2-46df-8864-0198ddcae33b None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:37.817637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2c0c9bb6-45e2-46df-8864-0198ddcae33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:37.899887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 173/336] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:46:37.935467 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182fe70-fa3a-4dba-afea-09fec41314a9 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:38.074024 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b182fe70-fa3a-4dba-afea-09fec41314a9 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:38.121519 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3c045-7ddc-4827-806e-d34940b98e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:38.122182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3c045-7ddc-4827-806e-d34940b98e9d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:38.122463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3c045-7ddc-4827-806e-d34940b98e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:38.122773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3c045-7ddc-4827-806e-d34940b98e9d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:38.139259 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3c045-7ddc-4827-806e-d34940b98e9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:38.144208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-1cd3c045-7ddc-4827-806e-d34940b98e9d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:38.144208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:38.144208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:38.144208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:38.144208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:38.144208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:38.144208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:38.144208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:38.144208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:38.144208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:38.144208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:38.144208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:38.144208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:38.144208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:38.144208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:38.144208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:38.145096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:38.145096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:38.145096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:38.145096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:38.145096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:38.145096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:38.145096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:38.145096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:38.145096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:38.145096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:38.145096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:38.145096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:38.145096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:38.145096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:38.145096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:38.145096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:38.145909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:38.145909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:38.145909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:38.145909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:38.145909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:38.145909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:38.145909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:38.145909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:38.145909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:38.145909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:38.145909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:38.145909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:38.146995 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 164/337] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:38.151602 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c9bb6-45e2-46df-8864-0198ddcae33b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:38.194130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:38.270877 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0c9bb6-45e2-46df-8864-0198ddcae33b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:38.311789 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150cfcb-d72d-4e64-b846-66f6f906ab06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:38.312481 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150cfcb-d72d-4e64-b846-66f6f906ab06 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:38.312763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150cfcb-d72d-4e64-b846-66f6f906ab06 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:38.313034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150cfcb-d72d-4e64-b846-66f6f906ab06 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:38.313754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a150cfcb-d72d-4e64-b846-66f6f906ab06 None admin] RBAC: Adding query filter params (name=member) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:38.314083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a150cfcb-d72d-4e64-b846-66f6f906ab06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:38.315521 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a150cfcb-d72d-4e64-b846-66f6f906ab06 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:38.329824 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 174/338] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:38.335964 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3c045-7ddc-4827-806e-d34940b98e9d None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:38.375296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:38.456492 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3c045-7ddc-4827-806e-d34940b98e9d None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:38.503686 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78bb8c-39d2-4298-ae8e-cbb814ac2f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:38.504303 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78bb8c-39d2-4298-ae8e-cbb814ac2f22 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:38.504631 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78bb8c-39d2-4298-ae8e-cbb814ac2f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:38.504912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a78bb8c-39d2-4298-ae8e-cbb814ac2f22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:38.588281 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a78bb8c-39d2-4298-ae8e-cbb814ac2f22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:38.589657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a78bb8c-39d2-4298-ae8e-cbb814ac2f22 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:38.623970 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:38.626402 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b46ade-110d-47db-ad1c-8a772daa65ee None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:38.626691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b46ade-110d-47db-ad1c-8a772daa65ee None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:38.626966 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b46ade-110d-47db-ad1c-8a772daa65ee None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:38.628316 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 175/339] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:38.633539 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69d661-063e-4b05-8d58-e6341bc5fa0b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:38.633825 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69d661-063e-4b05-8d58-e6341bc5fa0b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:38.634140 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e69d661-063e-4b05-8d58-e6341bc5fa0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:38.673736 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-7e69d661-063e-4b05-8d58-e6341bc5fa0b None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:38.674988 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7e69d661-063e-4b05-8d58-e6341bc5fa0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:38.723324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 165/340] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:46:38.732401 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78bb8c-39d2-4298-ae8e-cbb814ac2f22 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:38.826939 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 176/341] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:46:38.828842 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7bbff-9b94-4909-9133-fe00a519427c None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:38.828842 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7bbff-9b94-4909-9133-fe00a519427c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:38.828842 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7bbff-9b94-4909-9133-fe00a519427c None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:38.830006 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 177/342] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:38.833773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab82216-c902-4366-a678-a9bbc6705e12 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:38.834682 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab82216-c902-4366-a678-a9bbc6705e12 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:38.834961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab82216-c902-4366-a678-a9bbc6705e12 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:38.836841 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 178/343] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:38.845262 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6105cd-4581-4e63-869c-d70746f59686 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:38.845262 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6105cd-4581-4e63-869c-d70746f59686 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:38.845262 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6105cd-4581-4e63-869c-d70746f59686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:38.861742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9d6105cd-4581-4e63-869c-d70746f59686 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:38.863180 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9d6105cd-4581-4e63-869c-d70746f59686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:38.892246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:38.903136 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a78bb8c-39d2-4298-ae8e-cbb814ac2f22 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:38.954018 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 179/344] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:46:38.955182 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c970152-87b7-4ac8-afaf-14f2e35e5d04 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:38.955399 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c970152-87b7-4ac8-afaf-14f2e35e5d04 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:38.955679 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c970152-87b7-4ac8-afaf-14f2e35e5d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:38.974658 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f8626-a13b-4ac2-a0b0-337676edb177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:38.975319 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f8626-a13b-4ac2-a0b0-337676edb177 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:38.975599 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f8626-a13b-4ac2-a0b0-337676edb177 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:38.976427 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f8626-a13b-4ac2-a0b0-337676edb177 None admin] PATH_INFO: `/v3/projects/592a7a9f7334413584fa4b0d080c1f29` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:38.978613 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f8626-a13b-4ac2-a0b0-337676edb177 None admin] RBAC: Authorizing `identity:get_project(project_id=592a7a9f7334413584fa4b0d080c1f29)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:38.980132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f8626-a13b-4ac2-a0b0-337676edb177 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:38.982756 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 166/345] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:38 2022] GET /identity/v3/projects/592a7a9f7334413584fa4b0d080c1f29 => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:38.983546 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f8626-a13b-4ac2-a0b0-337676edb177 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:38.989786 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:38.992791 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5c970152-87b7-4ac8-afaf-14f2e35e5d04 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:38.994029 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5c970152-87b7-4ac8-afaf-14f2e35e5d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:39.078424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 180/346] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:39.086259 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c970152-87b7-4ac8-afaf-14f2e35e5d04 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:39.117411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f8626-a13b-4ac2-a0b0-337676edb177 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:39.160017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37650b68-755d-49d1-b87f-127258192dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:39.160663 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37650b68-755d-49d1-b87f-127258192dae None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:39.160948 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37650b68-755d-49d1-b87f-127258192dae None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:39.161225 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37650b68-755d-49d1-b87f-127258192dae None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:39.176509 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37650b68-755d-49d1-b87f-127258192dae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:39.184901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-37650b68-755d-49d1-b87f-127258192dae None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:46:39.184901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:39.184901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:39.184901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:39.184901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:39.184901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:39.184901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:39.184901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:39.184901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:39.184901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:39.184901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:39.184901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:39.184901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:39.184901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:39.184901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:39.184901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:39.185818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:39.185818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:39.185818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:39.185818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:39.185818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:39.185818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:39.185818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:39.185818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:39.185818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:39.185818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:39.185818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:39.185818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:39.185818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:39.185818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:39.185818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:39.185818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:39.187059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:39.187059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:39.187059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:39.187059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:39.187059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:39.187059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:39.187059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:39.187059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:39.187059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:39.187059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:39.187059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 08:46:39.187059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:39.187059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 167/347] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:39.187059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37650b68-755d-49d1-b87f-127258192dae None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:39.207983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:39.229973 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c970152-87b7-4ac8-afaf-14f2e35e5d04 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:39.264773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:39.273298 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74118d-b4cf-4d26-af5c-80c6179f13c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:39.274018 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74118d-b4cf-4d26-af5c-80c6179f13c3 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:39.274309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74118d-b4cf-4d26-af5c-80c6179f13c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:39.274753 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74118d-b4cf-4d26-af5c-80c6179f13c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:39.288114 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c74118d-b4cf-4d26-af5c-80c6179f13c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:39.295030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-5c74118d-b4cf-4d26-af5c-80c6179f13c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:39.295030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:39.295030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:39.295030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:39.295030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:39.295030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:39.295030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:39.295030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:39.295030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:39.295030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:39.295030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:39.295030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:39.295030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:39.295030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:39.295030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:39.295030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:39.297171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:39.297171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:39.297171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:39.297171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:39.297171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:39.297171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:39.297171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:39.297171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:39.297171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:39.297171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:39.297171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:39.297171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:39.297171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:39.297171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:39.297171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:39.297171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:39.298653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:39.298653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:39.298653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:39.298653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:39.298653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:39.298653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:39.298653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:39.298653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:39.298653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:39.298653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:39.298653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:39.298653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:39.298653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 181/348] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 2 08:46:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:39.298653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74118d-b4cf-4d26-af5c-80c6179f13c3 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:39.322839 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37650b68-755d-49d1-b87f-127258192dae None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:39.363225 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c284c-3834-4b0e-b9de-16694741b911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:39.363946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c284c-3834-4b0e-b9de-16694741b911 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:39.364225 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c284c-3834-4b0e-b9de-16694741b911 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:39.364499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c284c-3834-4b0e-b9de-16694741b911 None admin] PATH_INFO: `/v3/groups/d2b1b36d66ef4325a1f88fb1a1263b45` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:39.371204 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c284c-3834-4b0e-b9de-16694741b911 None admin] RBAC: Authorizing `identity:get_group(group_id=d2b1b36d66ef4325a1f88fb1a1263b45)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:39.372598 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c284c-3834-4b0e-b9de-16694741b911 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:39.375045 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 168/349] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 2 08:46:39 2022] GET /identity/v3/groups/d2b1b36d66ef4325a1f88fb1a1263b45 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:39.375822 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c284c-3834-4b0e-b9de-16694741b911 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:39.442247 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74118d-b4cf-4d26-af5c-80c6179f13c3 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:39.483814 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f464fc9-3f7f-4b9a-8b3f-0f717b303cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:39.484388 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c284c-3834-4b0e-b9de-16694741b911 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:39.484562 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f464fc9-3f7f-4b9a-8b3f-0f717b303cec None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:39.484729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f464fc9-3f7f-4b9a-8b3f-0f717b303cec None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:39.485012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f464fc9-3f7f-4b9a-8b3f-0f717b303cec None admin] PATH_INFO: `/v3/roles/member` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:39.496633 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f464fc9-3f7f-4b9a-8b3f-0f717b303cec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:39.503208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-7f464fc9-3f7f-4b9a-8b3f-0f717b303cec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:39.503208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:39.503208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:39.503208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:39.503208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:39.503208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:39.503208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:39.503208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:39.503208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:39.503208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:39.503208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:39.503208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:39.503208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:39.503208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:39.503208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:39.503208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:39.505335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:39.505335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:39.505335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:39.505335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:39.505335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:39.505335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:39.505335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:39.505335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:39.505335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:39.505335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:39.505335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:39.505335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:39.505335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:39.505335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:39.505335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:39.505335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:39.507746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:39.507746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:39.507746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:39.507746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:39.507746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:39.507746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:39.507746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:39.507746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:39.507746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:39.507746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:39.507746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:39.507746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:39.507746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 182/350] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:39.507746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f464fc9-3f7f-4b9a-8b3f-0f717b303cec None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:39.518724 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:39.520185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:39.529504 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62ac82-0c21-4774-bd94-a34b41369be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:39.530212 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62ac82-0c21-4774-bd94-a34b41369be4 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:39.530495 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62ac82-0c21-4774-bd94-a34b41369be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:39.530826 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c62ac82-0c21-4774-bd94-a34b41369be4 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:39.531565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c62ac82-0c21-4774-bd94-a34b41369be4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:39.531929 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c62ac82-0c21-4774-bd94-a34b41369be4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:39.533616 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c62ac82-0c21-4774-bd94-a34b41369be4 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:39.544559 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 169/351] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:39.545427 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62ac82-0c21-4774-bd94-a34b41369be4 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:39.631044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f464fc9-3f7f-4b9a-8b3f-0f717b303cec None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:39.670925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c62ac82-0c21-4774-bd94-a34b41369be4 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:39.688368 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d620057-af8d-4218-9cd9-56763b76fe79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:39.689005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d620057-af8d-4218-9cd9-56763b76fe79 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:39.689290 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d620057-af8d-4218-9cd9-56763b76fe79 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:39.689566 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d620057-af8d-4218-9cd9-56763b76fe79 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:39.690327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d620057-af8d-4218-9cd9-56763b76fe79 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:39.690632 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d620057-af8d-4218-9cd9-56763b76fe79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:39.692019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d620057-af8d-4218-9cd9-56763b76fe79 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:39.701236 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 183/352] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:46:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:39.701842 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d620057-af8d-4218-9cd9-56763b76fe79 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:39.719802 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88149f21-3900-480d-b5ba-dd4c5ccca657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:39.720583 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88149f21-3900-480d-b5ba-dd4c5ccca657 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:39.720929 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88149f21-3900-480d-b5ba-dd4c5ccca657 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:39.721277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88149f21-3900-480d-b5ba-dd4c5ccca657 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:39.722229 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88149f21-3900-480d-b5ba-dd4c5ccca657 None admin] RBAC: Adding query filter params (group.id=d2b1b36d66ef4325a1f88fb1a1263b45, role.id=68fe920c9be84b3aa1d35bdce554d96a, scope.project.id=592a7a9f7334413584fa4b0d080c1f29) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:39.722611 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88149f21-3900-480d-b5ba-dd4c5ccca657 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:39.724208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88149f21-3900-480d-b5ba-dd4c5ccca657 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:39.737348 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 170/353] 10.222.0.46 () {56 vars in 1376 bytes} [Tue Aug 2 08:46:39 2022] GET /identity/v3/role_assignments?group.id=d2b1b36d66ef4325a1f88fb1a1263b45&scope.project.id=592a7a9f7334413584fa4b0d080c1f29&role.id=68fe920c9be84b3aa1d35bdce554d96a => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:39.737348 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88149f21-3900-480d-b5ba-dd4c5ccca657 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:39.802249 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:39.843834 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:39.855420 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88149f21-3900-480d-b5ba-dd4c5ccca657 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:39.893766 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6748b0d-bde8-40b7-a009-7c0c27955801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:39.894821 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6748b0d-bde8-40b7-a009-7c0c27955801 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:39.895320 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6748b0d-bde8-40b7-a009-7c0c27955801 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:39.895860 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6748b0d-bde8-40b7-a009-7c0c27955801 None admin] PATH_INFO: `/v3/users/85dd4c5d75954e7abdfa21159112b03d` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:39.897343 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6748b0d-bde8-40b7-a009-7c0c27955801 None admin] RBAC: Authorizing `identity:get_user(user_id=85dd4c5d75954e7abdfa21159112b03d)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:39.900067 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6748b0d-bde8-40b7-a009-7c0c27955801 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:39.901996 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 171/354] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:46:39 2022] GET /identity/v3/users/85dd4c5d75954e7abdfa21159112b03d => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:39.902506 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d620057-af8d-4218-9cd9-56763b76fe79 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:39.903523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6748b0d-bde8-40b7-a009-7c0c27955801 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:39.994688 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a46b8c-6d8b-4402-b4d7-4294269c1b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:39.995343 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a46b8c-6d8b-4402-b4d7-4294269c1b04 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:39.995631 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a46b8c-6d8b-4402-b4d7-4294269c1b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:39.995913 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a46b8c-6d8b-4402-b4d7-4294269c1b04 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:39.996883 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a46b8c-6d8b-4402-b4d7-4294269c1b04 None admin] RBAC: Adding query filter params (name=member) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:39.997081 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a46b8c-6d8b-4402-b4d7-4294269c1b04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:39.998531 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a46b8c-6d8b-4402-b4d7-4294269c1b04 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:40.013691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 184/355] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:40.014586 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a46b8c-6d8b-4402-b4d7-4294269c1b04 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:40.026025 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6748b0d-bde8-40b7-a009-7c0c27955801 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:40.048713 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:40.064245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0537b6-e9ee-45e0-8864-4b45b7121e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:40.064879 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0537b6-e9ee-45e0-8864-4b45b7121e60 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:40.065165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0537b6-e9ee-45e0-8864-4b45b7121e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:40.065437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0537b6-e9ee-45e0-8864-4b45b7121e60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:40.154707 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0537b6-e9ee-45e0-8864-4b45b7121e60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:40.156107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0537b6-e9ee-45e0-8864-4b45b7121e60 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:40.179524 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a46b8c-6d8b-4402-b4d7-4294269c1b04 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:40.212799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:40.218261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4a83c-6226-4079-a0b1-bf7d579723d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:40.218891 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4a83c-6226-4079-a0b1-bf7d579723d4 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:40.219352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4a83c-6226-4079-a0b1-bf7d579723d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:40.219633 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf4a83c-6226-4079-a0b1-bf7d579723d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:40.220534 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4a83c-6226-4079-a0b1-bf7d579723d4 None admin] RBAC: Adding query filter params (role.id=3c439be3950947478195dcc08835a9b5, user.id=85dd4c5d75954e7abdfa21159112b03d, scope.system=all) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:40.220888 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4a83c-6226-4079-a0b1-bf7d579723d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:40.222413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf4a83c-6226-4079-a0b1-bf7d579723d4 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:40.236405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 185/356] 10.222.0.46 () {56 vars in 1308 bytes} [Tue Aug 2 08:46:40 2022] GET /identity/v3/role_assignments?user.id=85dd4c5d75954e7abdfa21159112b03d&scope.system=all&role.id=3c439be3950947478195dcc08835a9b5 => generated 510 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:40.237362 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4a83c-6226-4079-a0b1-bf7d579723d4 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:40.265944 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 172/357] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:46:40.267223 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975ab16-7d8a-44ac-916f-f2bcce237407 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:40.267517 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975ab16-7d8a-44ac-916f-f2bcce237407 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:40.267793 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975ab16-7d8a-44ac-916f-f2bcce237407 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:40.269064 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 173/358] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:40.273941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975ab16-7d8a-44ac-916f-f2bcce237407 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:40.323982 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:40.392070 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf4a83c-6226-4079-a0b1-bf7d579723d4 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:40.447474 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da8a2e-5f83-4752-9154-47ec1ee3677a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:40.448996 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da8a2e-5f83-4752-9154-47ec1ee3677a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:40.450351 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da8a2e-5f83-4752-9154-47ec1ee3677a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:40.451005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da8a2e-5f83-4752-9154-47ec1ee3677a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:40.492143 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975ab16-7d8a-44ac-916f-f2bcce237407 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:40.541310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2898177a-0a0d-4364-b0c1-d77cd37c42f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:40.541986 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898177a-0a0d-4364-b0c1-d77cd37c42f7 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:40.542269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898177a-0a0d-4364-b0c1-d77cd37c42f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:40.542544 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898177a-0a0d-4364-b0c1-d77cd37c42f7 None admin] PATH_INFO: `/v3/projects/592a7a9f7334413584fa4b0d080c1f29` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:40.544360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2898177a-0a0d-4364-b0c1-d77cd37c42f7 None admin] RBAC: Authorizing `identity:get_project(project_id=592a7a9f7334413584fa4b0d080c1f29)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:40.545801 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2898177a-0a0d-4364-b0c1-d77cd37c42f7 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:40.552108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 174/359] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:40 2022] GET /identity/v3/projects/592a7a9f7334413584fa4b0d080c1f29 => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:40.552108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5ecad-d862-4815-8e28-e0fd56bd2134 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:40.552108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5ecad-d862-4815-8e28-e0fd56bd2134 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:40.552108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5ecad-d862-4815-8e28-e0fd56bd2134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:40.562881 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:40.574502 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-2aa5ecad-d862-4815-8e28-e0fd56bd2134 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:40.576244 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-2aa5ecad-d862-4815-8e28-e0fd56bd2134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:40.577903 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da8a2e-5f83-4752-9154-47ec1ee3677a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:40.579913 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da8a2e-5f83-4752-9154-47ec1ee3677a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:40.667568 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 175/360] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:40.668441 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5ecad-d862-4815-8e28-e0fd56bd2134 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:40.689702 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:40.718628 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 186/361] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:46:40.720341 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004979a9-404a-46cc-9a5f-fa4dd3ed9a04 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:40.720635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004979a9-404a-46cc-9a5f-fa4dd3ed9a04 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:40.721512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004979a9-404a-46cc-9a5f-fa4dd3ed9a04 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:40.722862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 187/362] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:40.723906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004979a9-404a-46cc-9a5f-fa4dd3ed9a04 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:40.800081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5ecad-d862-4815-8e28-e0fd56bd2134 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:40.837048 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:40.844356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d0220-6d24-4117-8f9b-f3a1fb354f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:40.845013 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d0220-6d24-4117-8f9b-f3a1fb354f2f None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:40.845299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d0220-6d24-4117-8f9b-f3a1fb354f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:40.845575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d0220-6d24-4117-8f9b-f3a1fb354f2f None admin] PATH_INFO: `/v3/groups/d2b1b36d66ef4325a1f88fb1a1263b45` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:40.847596 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22d0220-6d24-4117-8f9b-f3a1fb354f2f None admin] RBAC: Authorizing `identity:get_group(group_id=d2b1b36d66ef4325a1f88fb1a1263b45)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:40.849047 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22d0220-6d24-4117-8f9b-f3a1fb354f2f None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:40.849710 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004979a9-404a-46cc-9a5f-fa4dd3ed9a04 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:40.850762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 176/363] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 2 08:46:40 2022] GET /identity/v3/groups/d2b1b36d66ef4325a1f88fb1a1263b45 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:40.851698 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d0220-6d24-4117-8f9b-f3a1fb354f2f None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:40.895273 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06581537-271e-4e2a-9112-40ef306aa7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:40.896464 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06581537-271e-4e2a-9112-40ef306aa7a6 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:40.897830 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06581537-271e-4e2a-9112-40ef306aa7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:40.898312 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06581537-271e-4e2a-9112-40ef306aa7a6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:40.913601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06581537-271e-4e2a-9112-40ef306aa7a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:40.919812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-06581537-271e-4e2a-9112-40ef306aa7a6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:40.919812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:40.919812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:40.919812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:40.919812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:40.919812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:40.919812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:40.919812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:40.919812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:40.919812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:40.919812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:40.919812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:40.919812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:40.919812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:40.919812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:40.919812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:40.920713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:40.920713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:40.920713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:40.920713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:40.920713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:40.920713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:40.920713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:40.920713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:40.920713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:40.920713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:40.920713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:40.920713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:40.920713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:40.920713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:40.920713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:40.920713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:40.921536 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:40.921536 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:40.921536 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:40.921536 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:40.921536 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:40.921536 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:40.921536 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:40.921536 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:40.921536 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:40.921536 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:40.921536 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:40.921536 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:40.922289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 188/364] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 2 08:46:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:40.925873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92519a0-e66b-4b2d-a7ec-8d3f0e3aa7c8 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:40.926443 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92519a0-e66b-4b2d-a7ec-8d3f0e3aa7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:40.926795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92519a0-e66b-4b2d-a7ec-8d3f0e3aa7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:40.954758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-a92519a0-e66b-4b2d-a7ec-8d3f0e3aa7c8 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:40.956316 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a92519a0-e66b-4b2d-a7ec-8d3f0e3aa7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:40.967022 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:40.983926 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d0220-6d24-4117-8f9b-f3a1fb354f2f None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:41.027204 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342867c-5a3e-4c6e-91a9-3d0ef36e4b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:41.028410 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342867c-5a3e-4c6e-91a9-3d0ef36e4b46 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:41.028410 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342867c-5a3e-4c6e-91a9-3d0ef36e4b46 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:41.028573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342867c-5a3e-4c6e-91a9-3d0ef36e4b46 None admin] PATH_INFO: `/v3/projects/f8bf83f1361247688208d846845c9d82` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:41.030292 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342867c-5a3e-4c6e-91a9-3d0ef36e4b46 None admin] RBAC: Authorizing `identity:get_project(project_id=f8bf83f1361247688208d846845c9d82)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:41.031972 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0342867c-5a3e-4c6e-91a9-3d0ef36e4b46 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:41.034234 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 177/365] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:40 2022] GET /identity/v3/projects/f8bf83f1361247688208d846845c9d82 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:41.034886 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342867c-5a3e-4c6e-91a9-3d0ef36e4b46 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:41.051169 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 189/366] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:41.052141 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92519a0-e66b-4b2d-a7ec-8d3f0e3aa7c8 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:41.163727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342867c-5a3e-4c6e-91a9-3d0ef36e4b46 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:41.166328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:41.190874 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92519a0-e66b-4b2d-a7ec-8d3f0e3aa7c8 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:41.211947 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03482341-a33f-4852-afa3-afa64d2a06b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:41.212575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03482341-a33f-4852-afa3-afa64d2a06b8 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:41.212850 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03482341-a33f-4852-afa3-afa64d2a06b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:41.213190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03482341-a33f-4852-afa3-afa64d2a06b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:41.215567 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03482341-a33f-4852-afa3-afa64d2a06b8 None admin] RBAC: Adding query filter params (group.id=d2b1b36d66ef4325a1f88fb1a1263b45, role.id=111e800ddb0a411c80664cd51ce13fc6, scope.project.id=592a7a9f7334413584fa4b0d080c1f29) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:41.215567 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03482341-a33f-4852-afa3-afa64d2a06b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:41.215757 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03482341-a33f-4852-afa3-afa64d2a06b8 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:41.221737 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 178/367] 10.222.0.46 () {56 vars in 1376 bytes} [Tue Aug 2 08:46:41 2022] GET /identity/v3/role_assignments?group.id=d2b1b36d66ef4325a1f88fb1a1263b45&scope.project.id=592a7a9f7334413584fa4b0d080c1f29&role.id=111e800ddb0a411c80664cd51ce13fc6 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:41.222472 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03482341-a33f-4852-afa3-afa64d2a06b8 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:41.254771 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:41.256730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ffe25-0d0b-48e7-bf16-644766b78f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:41.257910 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ffe25-0d0b-48e7-bf16-644766b78f6b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:41.258227 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ffe25-0d0b-48e7-bf16-644766b78f6b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:41.258502 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ffe25-0d0b-48e7-bf16-644766b78f6b None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:41.259246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ffe25-0d0b-48e7-bf16-644766b78f6b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:41.259555 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ffe25-0d0b-48e7-bf16-644766b78f6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:41.261135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ffe25-0d0b-48e7-bf16-644766b78f6b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:41.270864 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 190/368] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:46:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:41.271813 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ffe25-0d0b-48e7-bf16-644766b78f6b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:41.369480 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03482341-a33f-4852-afa3-afa64d2a06b8 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:41.391793 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ffe25-0d0b-48e7-bf16-644766b78f6b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:41.414868 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c0099-41fe-4d0c-8ce3-92c7ae47f0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:41.415518 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c0099-41fe-4d0c-8ce3-92c7ae47f0b9 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:41.416260 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c0099-41fe-4d0c-8ce3-92c7ae47f0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:41.416549 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c0099-41fe-4d0c-8ce3-92c7ae47f0b9 None admin] PATH_INFO: `/v3/groups/d2b1b36d66ef4325a1f88fb1a1263b45` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:41.418045 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c0099-41fe-4d0c-8ce3-92c7ae47f0b9 None admin] RBAC: Authorizing `identity:get_group(group_id=d2b1b36d66ef4325a1f88fb1a1263b45)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:41.420134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c0099-41fe-4d0c-8ce3-92c7ae47f0b9 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:41.423532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 179/369] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 2 08:46:41 2022] GET /identity/v3/groups/d2b1b36d66ef4325a1f88fb1a1263b45 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:41.423532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c0099-41fe-4d0c-8ce3-92c7ae47f0b9 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:41.439479 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492dc99-b13f-496e-a407-04389ed19008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:41.440129 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492dc99-b13f-496e-a407-04389ed19008 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:41.440412 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492dc99-b13f-496e-a407-04389ed19008 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:41.440797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492dc99-b13f-496e-a407-04389ed19008 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:41.454311 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b492dc99-b13f-496e-a407-04389ed19008 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:41.461930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-b492dc99-b13f-496e-a407-04389ed19008 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:46:41.461930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:41.461930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:41.461930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:41.461930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:41.461930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:41.461930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:41.461930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:41.461930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:41.461930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:41.461930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:41.461930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:41.461930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:41.461930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:41.461930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:41.461930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:41.462829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:41.462829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:41.462829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:41.462829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:41.462829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:41.462829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:41.462829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:41.462829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:41.462829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:41.462829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:41.462829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:41.462829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:41.462829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:41.462829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:41.462829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:41.462829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:41.464080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:41.464080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:41.464080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:41.464080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:41.464080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:41.464080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:41.464080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:41.464080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:41.464080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:41.464080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:41.464080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:46:41.464080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:41.464080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 191/370] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:46:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:41.464908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492dc99-b13f-496e-a407-04389ed19008 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:41.467209 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:41.528933 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:41.547519 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c0099-41fe-4d0c-8ce3-92c7ae47f0b9 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:41.591671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88334061-9cd3-4492-b27f-681e0a0b4795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:41.592606 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88334061-9cd3-4492-b27f-681e0a0b4795 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:41.592898 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88334061-9cd3-4492-b27f-681e0a0b4795 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:41.593178 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88334061-9cd3-4492-b27f-681e0a0b4795 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:41.595661 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492dc99-b13f-496e-a407-04389ed19008 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:41.641070 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f3e9ff-004b-45bf-a38d-807cff4d4e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:41.642119 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3e9ff-004b-45bf-a38d-807cff4d4e89 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:41.642648 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3e9ff-004b-45bf-a38d-807cff4d4e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:41.643948 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f3e9ff-004b-45bf-a38d-807cff4d4e89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:41.645548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f3e9ff-004b-45bf-a38d-807cff4d4e89 None admin] RBAC: Adding query filter params (group.id=d2b1b36d66ef4325a1f88fb1a1263b45, role.id=68fe920c9be84b3aa1d35bdce554d96a, scope.project.id=f8bf83f1361247688208d846845c9d82) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:41.646094 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f3e9ff-004b-45bf-a38d-807cff4d4e89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:41.648433 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f3e9ff-004b-45bf-a38d-807cff4d4e89 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:41.661789 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 192/371] 10.222.0.46 () {56 vars in 1376 bytes} [Tue Aug 2 08:46:41 2022] GET /identity/v3/role_assignments?group.id=d2b1b36d66ef4325a1f88fb1a1263b45&scope.project.id=f8bf83f1361247688208d846845c9d82&role.id=68fe920c9be84b3aa1d35bdce554d96a => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:41.663094 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f3e9ff-004b-45bf-a38d-807cff4d4e89 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:41.680402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88334061-9cd3-4492-b27f-681e0a0b4795 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:41.680402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88334061-9cd3-4492-b27f-681e0a0b4795 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:41.782712 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:41.790255 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:41.805778 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 180/372] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:46:41.814991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88334061-9cd3-4492-b27f-681e0a0b4795 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:41.839127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f3e9ff-004b-45bf-a38d-807cff4d4e89 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:41.934725 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88334061-9cd3-4492-b27f-681e0a0b4795 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:41.941861 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3697f1da-399e-4681-b7b4-2f60e91a7033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:41.942669 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697f1da-399e-4681-b7b4-2f60e91a7033 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:41.942955 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697f1da-399e-4681-b7b4-2f60e91a7033 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:41.943237 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3697f1da-399e-4681-b7b4-2f60e91a7033 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:41.943964 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3697f1da-399e-4681-b7b4-2f60e91a7033 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:41.944272 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3697f1da-399e-4681-b7b4-2f60e91a7033 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:41.947052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3697f1da-399e-4681-b7b4-2f60e91a7033 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:41.968613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 193/373] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:41.972774 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3697f1da-399e-4681-b7b4-2f60e91a7033 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:41.987852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280e92d-8f72-4f88-8622-4416d9621558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:41.989121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280e92d-8f72-4f88-8622-4416d9621558 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:41.989585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280e92d-8f72-4f88-8622-4416d9621558 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:41.990144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280e92d-8f72-4f88-8622-4416d9621558 None admin] PATH_INFO: `/v3/projects/f8bf83f1361247688208d846845c9d82` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:41.991821 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280e92d-8f72-4f88-8622-4416d9621558 None admin] RBAC: Authorizing `identity:get_project(project_id=f8bf83f1361247688208d846845c9d82)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:41.993503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280e92d-8f72-4f88-8622-4416d9621558 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:41.998635 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 181/374] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:41 2022] GET /identity/v3/projects/f8bf83f1361247688208d846845c9d82 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:42.003184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280e92d-8f72-4f88-8622-4416d9621558 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:42.100885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:42.163046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3697f1da-399e-4681-b7b4-2f60e91a7033 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:42.185574 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280e92d-8f72-4f88-8622-4416d9621558 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:42.219826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:42.232873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816387d-a67d-4b05-992a-8c394a86ed24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:42.233507 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816387d-a67d-4b05-992a-8c394a86ed24 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:42.233798 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816387d-a67d-4b05-992a-8c394a86ed24 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:42.234108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816387d-a67d-4b05-992a-8c394a86ed24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:42.249012 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e535c5-de30-4521-a7fe-61b876723e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:42.249669 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e535c5-de30-4521-a7fe-61b876723e46 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:42.249978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e535c5-de30-4521-a7fe-61b876723e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:42.250255 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e535c5-de30-4521-a7fe-61b876723e46 None admin] PATH_INFO: `/v3/groups/d2b1b36d66ef4325a1f88fb1a1263b45` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:42.251444 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e535c5-de30-4521-a7fe-61b876723e46 None admin] RBAC: Authorizing `identity:get_group(group_id=d2b1b36d66ef4325a1f88fb1a1263b45)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:42.252890 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e535c5-de30-4521-a7fe-61b876723e46 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:42.254867 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 182/375] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 2 08:46:42 2022] GET /identity/v3/groups/d2b1b36d66ef4325a1f88fb1a1263b45 => generated 226 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:42.260556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e535c5-de30-4521-a7fe-61b876723e46 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:42.373561 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e535c5-de30-4521-a7fe-61b876723e46 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:42.383346 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4816387d-a67d-4b05-992a-8c394a86ed24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:42.384839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4816387d-a67d-4b05-992a-8c394a86ed24 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:42.403033 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:42.415594 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa2dbb5-3707-46d8-b91a-7a6e35105494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:42.419989 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2dbb5-3707-46d8-b91a-7a6e35105494 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:42.419989 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2dbb5-3707-46d8-b91a-7a6e35105494 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:42.419989 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa2dbb5-3707-46d8-b91a-7a6e35105494 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:42.419989 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2dbb5-3707-46d8-b91a-7a6e35105494 None admin] RBAC: Adding query filter params (group.id=d2b1b36d66ef4325a1f88fb1a1263b45, role.id=111e800ddb0a411c80664cd51ce13fc6, scope.project.id=f8bf83f1361247688208d846845c9d82) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:42.419989 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2dbb5-3707-46d8-b91a-7a6e35105494 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:42.419989 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa2dbb5-3707-46d8-b91a-7a6e35105494 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:42.426196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 183/376] 10.222.0.46 () {56 vars in 1376 bytes} [Tue Aug 2 08:46:42 2022] GET /identity/v3/role_assignments?group.id=d2b1b36d66ef4325a1f88fb1a1263b45&scope.project.id=f8bf83f1361247688208d846845c9d82&role.id=111e800ddb0a411c80664cd51ce13fc6 => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:42.461811 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01fd91-46d5-41cf-9a62-bbe694a16168 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:42.462128 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01fd91-46d5-41cf-9a62-bbe694a16168 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:42.462406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f01fd91-46d5-41cf-9a62-bbe694a16168 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:42.464059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 184/377] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:46:42.469544 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5aefe-80b0-4c41-ad83-2742d3981e6b None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:42.469922 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5aefe-80b0-4c41-ad83-2742d3981e6b None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:42.470149 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5aefe-80b0-4c41-ad83-2742d3981e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:42.489029 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-28d5aefe-80b0-4c41-ad83-2742d3981e6b None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:42.490312 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-28d5aefe-80b0-4c41-ad83-2742d3981e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:42.602838 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 194/378] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:46:42.611452 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 185/379] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:46:42.622771 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816387d-a67d-4b05-992a-8c394a86ed24 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:42.638857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5aefe-80b0-4c41-ad83-2742d3981e6b None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:42.667280 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:42.755498 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816387d-a67d-4b05-992a-8c394a86ed24 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:42.798312 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24fe0eb-4ce4-46b1-b5d4-3afffab614f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:42.799255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24fe0eb-4ce4-46b1-b5d4-3afffab614f0 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:42.799543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24fe0eb-4ce4-46b1-b5d4-3afffab614f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:42.799823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24fe0eb-4ce4-46b1-b5d4-3afffab614f0 None admin] PATH_INFO: `/v3/projects/2a92b68d350c4f7998c221772fb56b4a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:42.801236 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24fe0eb-4ce4-46b1-b5d4-3afffab614f0 None admin] RBAC: Authorizing `identity:get_project(project_id=2a92b68d350c4f7998c221772fb56b4a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:42.802815 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24fe0eb-4ce4-46b1-b5d4-3afffab614f0 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:42.805566 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 195/380] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:42 2022] GET /identity/v3/projects/2a92b68d350c4f7998c221772fb56b4a => generated 344 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:42.811453 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24fe0eb-4ce4-46b1-b5d4-3afffab614f0 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:42.860067 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5aefe-80b0-4c41-ad83-2742d3981e6b None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:42.880766 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:42.909101 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9becdd59-dd9e-4548-b3f1-c650d8564f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:42.909101 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becdd59-dd9e-4548-b3f1-c650d8564f47 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:42.909101 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becdd59-dd9e-4548-b3f1-c650d8564f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:42.909101 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becdd59-dd9e-4548-b3f1-c650d8564f47 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:42.914717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9becdd59-dd9e-4548-b3f1-c650d8564f47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:42.920486 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-9becdd59-dd9e-4548-b3f1-c650d8564f47 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:42.920486 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:42.920486 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:42.920486 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:42.920486 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:42.920486 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:42.920486 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:42.920486 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:42.920486 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:42.920486 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:42.920486 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:42.920486 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:42.920486 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:42.920486 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:42.920486 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:42.920486 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:42.921402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:42.921402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:42.921402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:42.921402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:42.921402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:42.921402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:42.921402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:42.921402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:42.921402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:42.921402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:42.921402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:42.921402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:42.921402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:42.921402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:42.921402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:42.921402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:42.922256 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:42.922256 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:42.922256 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:42.922256 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:42.922256 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:42.922256 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:42.922256 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:42.922256 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:42.922256 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:42.922256 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:42.922256 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:42.922256 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:42.922946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 186/381] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:42.928156 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9becdd59-dd9e-4548-b3f1-c650d8564f47 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:43.018317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:43.030214 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24fe0eb-4ce4-46b1-b5d4-3afffab614f0 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:43.102621 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3a3d8-b4a5-4089-96f8-bb6acad1dc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:43.103668 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3a3d8-b4a5-4089-96f8-bb6acad1dc60 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:43.104258 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3a3d8-b4a5-4089-96f8-bb6acad1dc60 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:43.104389 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9becdd59-dd9e-4548-b3f1-c650d8564f47 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:43.105210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3a3d8-b4a5-4089-96f8-bb6acad1dc60 None admin] PATH_INFO: `/v3/groups/723bcb65539446dcba0121b165a90e9c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:43.117142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3a3d8-b4a5-4089-96f8-bb6acad1dc60 None admin] RBAC: Authorizing `identity:get_group(group_id=723bcb65539446dcba0121b165a90e9c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:43.118955 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3a3d8-b4a5-4089-96f8-bb6acad1dc60 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:43.121281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 196/382] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 2 08:46:42 2022] GET /identity/v3/groups/723bcb65539446dcba0121b165a90e9c => generated 229 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:43.126693 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3a3d8-b4a5-4089-96f8-bb6acad1dc60 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:43.146147 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005b7ce-8bf3-4fff-99f1-699f5be91053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:43.146785 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005b7ce-8bf3-4fff-99f1-699f5be91053 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:43.147066 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005b7ce-8bf3-4fff-99f1-699f5be91053 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:43.147338 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005b7ce-8bf3-4fff-99f1-699f5be91053 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:43.148590 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005b7ce-8bf3-4fff-99f1-699f5be91053 None admin] RBAC: Adding query filter params (name=member) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:43.148895 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005b7ce-8bf3-4fff-99f1-699f5be91053 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:43.150315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005b7ce-8bf3-4fff-99f1-699f5be91053 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:43.162484 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 187/383] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:43.171230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005b7ce-8bf3-4fff-99f1-699f5be91053 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:43.220097 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:43.284150 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005b7ce-8bf3-4fff-99f1-699f5be91053 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:43.317334 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3a3d8-b4a5-4089-96f8-bb6acad1dc60 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:43.319994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ace20-0445-491c-9b89-8aeb39d14512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:43.320818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ace20-0445-491c-9b89-8aeb39d14512 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:43.321249 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ace20-0445-491c-9b89-8aeb39d14512 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:43.321655 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ace20-0445-491c-9b89-8aeb39d14512 None admin] PATH_INFO: `/v3/groups/d2b1b36d66ef4325a1f88fb1a1263b45` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:43.323251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ace20-0445-491c-9b89-8aeb39d14512 None admin] RBAC: Authorizing `identity:get_group(group_id=d2b1b36d66ef4325a1f88fb1a1263b45)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:43.324806 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ace20-0445-491c-9b89-8aeb39d14512 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:43.326599 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 188/384] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 2 08:46:43 2022] GET /identity/v3/groups/d2b1b36d66ef4325a1f88fb1a1263b45 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:43.331807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ace20-0445-491c-9b89-8aeb39d14512 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:43.379950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab8485-ff55-4948-951b-d60644c3d5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:43.380740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab8485-ff55-4948-951b-d60644c3d5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:43.381127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab8485-ff55-4948-951b-d60644c3d5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:43.381513 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab8485-ff55-4948-951b-d60644c3d5c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:43.382515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab8485-ff55-4948-951b-d60644c3d5c1 None admin] RBAC: Adding query filter params (group.id=723bcb65539446dcba0121b165a90e9c, role.id=8ce0850e15804f3d811e88cc9841e673, scope.project.id=2a92b68d350c4f7998c221772fb56b4a) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:43.382936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab8485-ff55-4948-951b-d60644c3d5c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:43.384739 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab8485-ff55-4948-951b-d60644c3d5c1 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:43.403821 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 197/385] 10.222.0.46 () {56 vars in 1376 bytes} [Tue Aug 2 08:46:43 2022] GET /identity/v3/role_assignments?group.id=723bcb65539446dcba0121b165a90e9c&scope.project.id=2a92b68d350c4f7998c221772fb56b4a&role.id=8ce0850e15804f3d811e88cc9841e673 => generated 260 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:43.447951 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ace20-0445-491c-9b89-8aeb39d14512 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:43.458287 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:43.486884 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518bb8f-ac1e-40bc-98b5-4635507cbf57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:43.491021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518bb8f-ac1e-40bc-98b5-4635507cbf57 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:43.491021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518bb8f-ac1e-40bc-98b5-4635507cbf57 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:43.491021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518bb8f-ac1e-40bc-98b5-4635507cbf57 None admin] PATH_INFO: `/v3/projects/592a7a9f7334413584fa4b0d080c1f29` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:43.497488 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518bb8f-ac1e-40bc-98b5-4635507cbf57 None admin] RBAC: Authorizing `identity:get_project(project_id=592a7a9f7334413584fa4b0d080c1f29)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:43.499194 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518bb8f-ac1e-40bc-98b5-4635507cbf57 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:43.501511 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 189/386] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:43 2022] GET /identity/v3/projects/592a7a9f7334413584fa4b0d080c1f29 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:43.508824 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab8485-ff55-4948-951b-d60644c3d5c1 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:43.598507 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:43.767954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab8485-ff55-4948-951b-d60644c3d5c1 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:43.828463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c084da-f02b-4034-97d6-0d63a7900d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:43.829157 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c084da-f02b-4034-97d6-0d63a7900d54 None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:43.829443 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c084da-f02b-4034-97d6-0d63a7900d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:43.829725 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c084da-f02b-4034-97d6-0d63a7900d54 None admin] PATH_INFO: `/v3/projects/592a7a9f7334413584fa4b0d080c1f29/groups/d2b1b36d66ef4325a1f88fb1a1263b45/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:43.835738 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c084da-f02b-4034-97d6-0d63a7900d54 None admin] RBAC: Authorizing `identity:create_grant(project_id=592a7a9f7334413584fa4b0d080c1f29, group_id=d2b1b36d66ef4325a1f88fb1a1263b45, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:43.837276 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c084da-f02b-4034-97d6-0d63a7900d54 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:43.945378 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 198/387] 10.222.0.46 () {58 vars in 1442 bytes} [Tue Aug 2 08:46:43 2022] PUT /identity/v3/projects/592a7a9f7334413584fa4b0d080c1f29/groups/d2b1b36d66ef4325a1f88fb1a1263b45/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:46:43.962903 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e84e05-20b5-4ee2-8916-abfd1df37aed None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:43.962903 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e84e05-20b5-4ee2-8916-abfd1df37aed None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:43.962903 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e84e05-20b5-4ee2-8916-abfd1df37aed None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:43.962903 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 190/388] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:43.973516 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840a01d-fb86-4c23-a319-32aecb9f673c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:43.973516 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840a01d-fb86-4c23-a319-32aecb9f673c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:43.973516 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840a01d-fb86-4c23-a319-32aecb9f673c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:44.004030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9840a01d-fb86-4c23-a319-32aecb9f673c None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:44.005654 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9840a01d-fb86-4c23-a319-32aecb9f673c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:44.128283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 199/389] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:44.163680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e84e05-20b5-4ee2-8916-abfd1df37aed None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:44.348373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e84e05-20b5-4ee2-8916-abfd1df37aed None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:44.388096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd11e8b0-7dff-46ab-a34f-5b0d517682ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:44.388676 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11e8b0-7dff-46ab-a34f-5b0d517682ad None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:44.388959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11e8b0-7dff-46ab-a34f-5b0d517682ad None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:44.389240 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd11e8b0-7dff-46ab-a34f-5b0d517682ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:44.404839 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd11e8b0-7dff-46ab-a34f-5b0d517682ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:44.409916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-cd11e8b0-7dff-46ab-a34f-5b0d517682ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:44.409916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:44.409916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:44.409916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:44.409916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:44.409916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:44.409916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:44.409916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:44.409916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:44.409916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:44.409916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:44.409916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:44.409916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:44.409916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:44.409916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:44.409916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:44.410857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:44.410857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:44.410857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:44.410857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:44.410857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:44.410857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:44.410857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:44.410857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:44.410857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:44.410857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:44.410857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:44.410857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:44.410857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:44.410857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:44.410857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:44.410857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:44.411699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:44.411699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:44.411699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:44.411699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:44.411699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:44.411699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:44.411699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:44.411699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:44.411699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:44.411699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:44.411699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:44.411699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:44.411699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 191/390] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 2 08:46:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:44.418889 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840a01d-fb86-4c23-a319-32aecb9f673c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:44.437031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:44.508188 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d01919-8a42-453d-8f1d-1f888fe3f1a4 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:44.508529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d01919-8a42-453d-8f1d-1f888fe3f1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:44.508754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d01919-8a42-453d-8f1d-1f888fe3f1a4 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:44.510310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 192/391] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:46:44.515244 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee11ffd-11a6-48ba-a7a0-513a758cc519 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:44.515538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee11ffd-11a6-48ba-a7a0-513a758cc519 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:44.515819 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee11ffd-11a6-48ba-a7a0-513a758cc519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:44.551034 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-bee11ffd-11a6-48ba-a7a0-513a758cc519 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:44.552245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-bee11ffd-11a6-48ba-a7a0-513a758cc519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:44.569428 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840a01d-fb86-4c23-a319-32aecb9f673c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:44.613241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:44.638751 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0903d802-4395-4c4d-b9fa-c8d04c3da058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:44.639309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903d802-4395-4c4d-b9fa-c8d04c3da058 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:44.639592 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903d802-4395-4c4d-b9fa-c8d04c3da058 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:44.639869 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0903d802-4395-4c4d-b9fa-c8d04c3da058 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:44.640603 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0903d802-4395-4c4d-b9fa-c8d04c3da058 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:44.640910 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0903d802-4395-4c4d-b9fa-c8d04c3da058 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:44.642448 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0903d802-4395-4c4d-b9fa-c8d04c3da058 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:44.660226 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 200/392] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:46:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:44.670213 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0903d802-4395-4c4d-b9fa-c8d04c3da058 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:44.696013 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 193/393] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:44.721055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee11ffd-11a6-48ba-a7a0-513a758cc519 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:44.808350 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:44.849013 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0903d802-4395-4c4d-b9fa-c8d04c3da058 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:44.952557 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee11ffd-11a6-48ba-a7a0-513a758cc519 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:44.954517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c913c-f45c-4f77-82e7-2834e836a7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:44.955183 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c913c-f45c-4f77-82e7-2834e836a7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:44.955467 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c913c-f45c-4f77-82e7-2834e836a7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:44.955745 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c913c-f45c-4f77-82e7-2834e836a7f9 None admin] PATH_INFO: `/v3/groups/d2b1b36d66ef4325a1f88fb1a1263b45` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:44.956964 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c913c-f45c-4f77-82e7-2834e836a7f9 None admin] RBAC: Authorizing `identity:get_group(group_id=d2b1b36d66ef4325a1f88fb1a1263b45)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:44.965278 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c913c-f45c-4f77-82e7-2834e836a7f9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:44.967404 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 201/394] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 2 08:46:44 2022] GET /identity/v3/groups/d2b1b36d66ef4325a1f88fb1a1263b45 => generated 226 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:44.979301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c913c-f45c-4f77-82e7-2834e836a7f9 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:45.029923 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abc8a5-d9c9-465c-a0ce-920d30981a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:45.030563 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abc8a5-d9c9-465c-a0ce-920d30981a00 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:45.030856 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abc8a5-d9c9-465c-a0ce-920d30981a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:45.031121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5abc8a5-d9c9-465c-a0ce-920d30981a00 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:45.053414 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5abc8a5-d9c9-465c-a0ce-920d30981a00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:45.058505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-d5abc8a5-d9c9-465c-a0ce-920d30981a00 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:45.058505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:45.058505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:45.058505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:45.058505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:45.058505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:45.058505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:45.058505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:45.058505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:45.058505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:45.058505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:45.058505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:45.058505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:45.058505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:45.058505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:45.058505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:45.059406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:45.059406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:45.059406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:45.059406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:45.059406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:45.059406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:45.059406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:45.059406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:45.059406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:45.059406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:45.059406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:45.059406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:45.059406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:45.059406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:45.059406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:45.059406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:45.060613 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:45.060613 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:45.060613 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:45.060613 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:45.060613 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:45.060613 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:45.060613 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:45.060613 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:45.060613 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:45.060613 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:45.060613 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:45.060613 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:45.060613 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 194/395] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:46:45.065635 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abc8a5-d9c9-465c-a0ce-920d30981a00 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:45.096938 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:45.181514 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5abc8a5-d9c9-465c-a0ce-920d30981a00 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:45.193687 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c913c-f45c-4f77-82e7-2834e836a7f9 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:45.222896 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0dc188-9646-49b2-a3fa-03e4474eb9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:45.223514 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0dc188-9646-49b2-a3fa-03e4474eb9bf None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:45.223797 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0dc188-9646-49b2-a3fa-03e4474eb9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:45.224072 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0dc188-9646-49b2-a3fa-03e4474eb9bf None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:45.224792 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0dc188-9646-49b2-a3fa-03e4474eb9bf None admin] RBAC: Adding query filter params (name=member) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:45.225097 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0dc188-9646-49b2-a3fa-03e4474eb9bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:45.227151 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0dc188-9646-49b2-a3fa-03e4474eb9bf None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:45.245256 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 195/396] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:45.253438 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0dc188-9646-49b2-a3fa-03e4474eb9bf None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:45.266586 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0dac0f-87b9-46f1-b37e-94d688c48b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:45.267335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0dac0f-87b9-46f1-b37e-94d688c48b41 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:45.267588 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0dac0f-87b9-46f1-b37e-94d688c48b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:45.267864 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0dac0f-87b9-46f1-b37e-94d688c48b41 None admin] PATH_INFO: `/v3/projects/592a7a9f7334413584fa4b0d080c1f29` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:45.269302 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0dac0f-87b9-46f1-b37e-94d688c48b41 None admin] RBAC: Authorizing `identity:get_project(project_id=592a7a9f7334413584fa4b0d080c1f29)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:45.270762 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0dac0f-87b9-46f1-b37e-94d688c48b41 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:45.275086 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 202/397] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:44 2022] GET /identity/v3/projects/592a7a9f7334413584fa4b0d080c1f29 => generated 298 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:45.285724 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fc4e8-b3c0-42bf-918d-fc2a29ba3b11 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:45.286086 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fc4e8-b3c0-42bf-918d-fc2a29ba3b11 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:45.286372 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03fc4e8-b3c0-42bf-918d-fc2a29ba3b11 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:45.287772 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 203/398] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:45.289026 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03fc4e8-b3c0-42bf-918d-fc2a29ba3b11 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:45.311656 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:45.377267 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:45.398452 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0dc188-9646-49b2-a3fa-03e4474eb9bf None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:45.427786 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03fc4e8-b3c0-42bf-918d-fc2a29ba3b11 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:45.454392 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca16973-10ff-4d2c-89b4-1084a662683e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:45.454961 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca16973-10ff-4d2c-89b4-1084a662683e None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:45.455242 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca16973-10ff-4d2c-89b4-1084a662683e None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:45.455517 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca16973-10ff-4d2c-89b4-1084a662683e None admin] PATH_INFO: `/v3/groups/d2b1b36d66ef4325a1f88fb1a1263b45` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:45.457664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca16973-10ff-4d2c-89b4-1084a662683e None admin] RBAC: Authorizing `identity:get_group(group_id=d2b1b36d66ef4325a1f88fb1a1263b45)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:45.459455 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca16973-10ff-4d2c-89b4-1084a662683e None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:45.465963 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 196/399] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 2 08:46:45 2022] GET /identity/v3/groups/d2b1b36d66ef4325a1f88fb1a1263b45 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:45.469445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20063f6e-ef5d-45b1-a80b-2974028cd0bd None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:45.469902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20063f6e-ef5d-45b1-a80b-2974028cd0bd None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:45.470363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20063f6e-ef5d-45b1-a80b-2974028cd0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:45.479276 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c45e021-f798-4c8a-affb-21a82ae5ebea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:45.480281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45e021-f798-4c8a-affb-21a82ae5ebea None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:45.480824 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45e021-f798-4c8a-affb-21a82ae5ebea None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:45.481316 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c45e021-f798-4c8a-affb-21a82ae5ebea None admin] PATH_INFO: `/v3/projects/592a7a9f7334413584fa4b0d080c1f29/groups/d2b1b36d66ef4325a1f88fb1a1263b45/roles/111e800ddb0a411c80664cd51ce13fc6` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:45.483887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c45e021-f798-4c8a-affb-21a82ae5ebea None admin] RBAC: Authorizing `identity:create_grant(project_id=592a7a9f7334413584fa4b0d080c1f29, group_id=d2b1b36d66ef4325a1f88fb1a1263b45, role_id=111e800ddb0a411c80664cd51ce13fc6)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:45.485560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c45e021-f798-4c8a-affb-21a82ae5ebea None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:45.488665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-20063f6e-ef5d-45b1-a80b-2974028cd0bd None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:45.489923 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-20063f6e-ef5d-45b1-a80b-2974028cd0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:45.563958 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 197/400] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:45.564958 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20063f6e-ef5d-45b1-a80b-2974028cd0bd None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:45.566515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 204/401] 10.222.0.46 () {58 vars in 1442 bytes} [Tue Aug 2 08:46:45 2022] PUT /identity/v3/projects/592a7a9f7334413584fa4b0d080c1f29/groups/d2b1b36d66ef4325a1f88fb1a1263b45/roles/111e800ddb0a411c80664cd51ce13fc6 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:46:45.586410 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c45e021-f798-4c8a-affb-21a82ae5ebea None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:45.673089 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:45.695209 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20063f6e-ef5d-45b1-a80b-2974028cd0bd None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:45.738458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb5046-c943-4a89-95a9-421d63890ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:45.739873 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb5046-c943-4a89-95a9-421d63890ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:45.740389 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb5046-c943-4a89-95a9-421d63890ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:45.740882 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb5046-c943-4a89-95a9-421d63890ac4 None admin] PATH_INFO: `/v3/projects/f8bf83f1361247688208d846845c9d82` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:45.743372 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb5046-c943-4a89-95a9-421d63890ac4 None admin] RBAC: Authorizing `identity:get_project(project_id=f8bf83f1361247688208d846845c9d82)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:45.745342 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb5046-c943-4a89-95a9-421d63890ac4 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:45.747672 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 198/402] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:45 2022] GET /identity/v3/projects/f8bf83f1361247688208d846845c9d82 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:45.754604 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb5046-c943-4a89-95a9-421d63890ac4 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:45.767062 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:45.800717 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c45e021-f798-4c8a-affb-21a82ae5ebea None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:45.840055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296e322-2100-45de-8108-ca1b10df9156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:45.840727 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296e322-2100-45de-8108-ca1b10df9156 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:45.841439 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296e322-2100-45de-8108-ca1b10df9156 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:45.841720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296e322-2100-45de-8108-ca1b10df9156 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:45.855999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d296e322-2100-45de-8108-ca1b10df9156 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:45.862726 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-d296e322-2100-45de-8108-ca1b10df9156 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:45.862726 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:45.862726 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:45.862726 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:45.862726 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:45.862726 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:45.862726 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:45.862726 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:45.862726 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:45.862726 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:45.862726 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:45.862726 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:45.862726 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:45.862726 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:45.862726 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:45.862726 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:45.863653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:45.863653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:45.863653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:45.863653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:45.863653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:45.863653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:45.863653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:45.863653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:45.863653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:45.863653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:45.863653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:45.863653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:45.863653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:45.863653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:45.863653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:45.863653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:45.864463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:45.864463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:45.864463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:45.864463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:45.864463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:45.864463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:45.864463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:45.864463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:45.864463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:45.864463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:45.864463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:45.864463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:45.865070 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 205/403] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 2 08:46:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:45.871507 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296e322-2100-45de-8108-ca1b10df9156 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:45.894345 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb5046-c943-4a89-95a9-421d63890ac4 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:45.954720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4044a8c3-b14c-4618-b355-fb6471b3ad9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:45.955405 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044a8c3-b14c-4618-b355-fb6471b3ad9d None admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:45.955687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044a8c3-b14c-4618-b355-fb6471b3ad9d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:45.955963 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4044a8c3-b14c-4618-b355-fb6471b3ad9d None admin] PATH_INFO: `/v3/projects/f8bf83f1361247688208d846845c9d82/groups/d2b1b36d66ef4325a1f88fb1a1263b45/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:45.958845 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4044a8c3-b14c-4618-b355-fb6471b3ad9d None admin] RBAC: Authorizing `identity:create_grant(project_id=f8bf83f1361247688208d846845c9d82, group_id=d2b1b36d66ef4325a1f88fb1a1263b45, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:45.960321 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4044a8c3-b14c-4618-b355-fb6471b3ad9d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:46.026909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296e322-2100-45de-8108-ca1b10df9156 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:46.057468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:46.074340 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 199/404] 10.222.0.46 () {58 vars in 1442 bytes} [Tue Aug 2 08:46:45 2022] PUT /identity/v3/projects/f8bf83f1361247688208d846845c9d82/groups/d2b1b36d66ef4325a1f88fb1a1263b45/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:46:46.074340 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1eb85-33c2-4163-a123-8c4efa6be403 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:46.074340 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1eb85-33c2-4163-a123-8c4efa6be403 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:46.074340 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e1eb85-33c2-4163-a123-8c4efa6be403 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:46.074340 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 200/405] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:46.076439 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a11579-8ede-490a-8e79-2f55467c2256 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:46.076826 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a11579-8ede-490a-8e79-2f55467c2256 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:46.077205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a11579-8ede-490a-8e79-2f55467c2256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:46.079095 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b60843-4095-44f2-8130-6a28db98d659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:46.079942 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b60843-4095-44f2-8130-6a28db98d659 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:46.080361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b60843-4095-44f2-8130-6a28db98d659 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:46.080835 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b60843-4095-44f2-8130-6a28db98d659 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:46.081737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b60843-4095-44f2-8130-6a28db98d659 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:46.082210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b60843-4095-44f2-8130-6a28db98d659 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:46.083856 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b60843-4095-44f2-8130-6a28db98d659 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:46.099187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 206/406] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:46:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:46.105798 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b60843-4095-44f2-8130-6a28db98d659 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:46.126468 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-28a11579-8ede-490a-8e79-2f55467c2256 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:46.130068 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-28a11579-8ede-490a-8e79-2f55467c2256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:46.150287 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:46.274024 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 201/407] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:46.284561 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b60843-4095-44f2-8130-6a28db98d659 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:46.298613 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a11579-8ede-490a-8e79-2f55467c2256 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:46.331462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700f2d5-343f-4d7a-abc5-9d1ab5829fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:46.332118 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700f2d5-343f-4d7a-abc5-9d1ab5829fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:46.332397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700f2d5-343f-4d7a-abc5-9d1ab5829fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:46.332671 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700f2d5-343f-4d7a-abc5-9d1ab5829fc5 None admin] PATH_INFO: `/v3/groups/d2b1b36d66ef4325a1f88fb1a1263b45` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:46.333823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700f2d5-343f-4d7a-abc5-9d1ab5829fc5 None admin] RBAC: Authorizing `identity:get_group(group_id=d2b1b36d66ef4325a1f88fb1a1263b45)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:46.336005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a700f2d5-343f-4d7a-abc5-9d1ab5829fc5 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:46.338136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 207/408] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 2 08:46:46 2022] GET /identity/v3/groups/d2b1b36d66ef4325a1f88fb1a1263b45 => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:46.348133 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700f2d5-343f-4d7a-abc5-9d1ab5829fc5 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:46.451165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a11579-8ede-490a-8e79-2f55467c2256 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:46.471648 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:46.492849 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d6bfe-2a1a-4427-b33b-13f19cfc613b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:46.494332 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d6bfe-2a1a-4427-b33b-13f19cfc613b None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:46.494618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d6bfe-2a1a-4427-b33b-13f19cfc613b None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:46.494896 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d6bfe-2a1a-4427-b33b-13f19cfc613b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:46.507781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d6bfe-2a1a-4427-b33b-13f19cfc613b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:46.513288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-4d5d6bfe-2a1a-4427-b33b-13f19cfc613b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:46:46.513288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:46.513288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:46.513288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:46.513288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:46.513288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:46.513288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:46.513288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:46.513288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:46.513288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:46.513288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:46.513288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:46.513288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:46.513288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:46.513288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:46.513288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:46.514208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:46.514208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:46.514208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:46.514208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:46.514208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:46.514208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:46.514208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:46.514208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:46.514208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:46.514208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:46.514208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:46.514208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:46.514208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:46.514208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:46.514208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:46.514208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:46.515002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:46.515002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:46.515002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:46.515002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:46.515002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:46.515002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:46.515002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:46.515002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:46.515002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:46.515002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:46.515002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:46:46.515002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:46.518684 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 202/409] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:46:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:46:46.521364 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d6bfe-2a1a-4427-b33b-13f19cfc613b None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:46.525169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:46.560163 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700f2d5-343f-4d7a-abc5-9d1ab5829fc5 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:46.620508 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a13b96e-2085-4e87-9e5d-a4ca3b5272d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:46.621211 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13b96e-2085-4e87-9e5d-a4ca3b5272d3 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:46.621501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13b96e-2085-4e87-9e5d-a4ca3b5272d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:46.621783 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a13b96e-2085-4e87-9e5d-a4ca3b5272d3 None admin] PATH_INFO: `/v3/projects/f8bf83f1361247688208d846845c9d82` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:46.622985 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13b96e-2085-4e87-9e5d-a4ca3b5272d3 None admin] RBAC: Authorizing `identity:get_project(project_id=f8bf83f1361247688208d846845c9d82)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:46.624440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a13b96e-2085-4e87-9e5d-a4ca3b5272d3 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:46.626500 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 208/410] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:46 2022] GET /identity/v3/projects/f8bf83f1361247688208d846845c9d82 => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:46.627646 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5773c7d-6a8e-4897-9ee1-c01f56775199 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:46.627933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5773c7d-6a8e-4897-9ee1-c01f56775199 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:46.628212 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5773c7d-6a8e-4897-9ee1-c01f56775199 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:46.633838 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 209/411] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:46.634741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5773c7d-6a8e-4897-9ee1-c01f56775199 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:46.657292 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d6bfe-2a1a-4427-b33b-13f19cfc613b None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:46.701636 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543d1671-dca0-4273-b639-810393ea64e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:46.702291 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d1671-dca0-4273-b639-810393ea64e2 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:46.702580 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d1671-dca0-4273-b639-810393ea64e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:46.702917 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d1671-dca0-4273-b639-810393ea64e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:46.704250 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d1671-dca0-4273-b639-810393ea64e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:46.704566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d1671-dca0-4273-b639-810393ea64e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:46.706022 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543d1671-dca0-4273-b639-810393ea64e2 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:46.721990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 203/412] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:46.721990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdab443-f151-4709-8a20-c293ceaf473f None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:46.721990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdab443-f151-4709-8a20-c293ceaf473f None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:46.721990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdab443-f151-4709-8a20-c293ceaf473f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:46.743638 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-4bdab443-f151-4709-8a20-c293ceaf473f None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:46.745476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4bdab443-f151-4709-8a20-c293ceaf473f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:46.751989 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5773c7d-6a8e-4897-9ee1-c01f56775199 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:46.793244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:46.800645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523124c2-2996-4662-bc38-4c5ccfb379e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:46.802059 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523124c2-2996-4662-bc38-4c5ccfb379e3 None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:46.802494 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523124c2-2996-4662-bc38-4c5ccfb379e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:46.803236 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523124c2-2996-4662-bc38-4c5ccfb379e3 None admin] PATH_INFO: `/v3/projects/f8bf83f1361247688208d846845c9d82/groups/d2b1b36d66ef4325a1f88fb1a1263b45/roles/111e800ddb0a411c80664cd51ce13fc6` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:46.806694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523124c2-2996-4662-bc38-4c5ccfb379e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=f8bf83f1361247688208d846845c9d82, group_id=d2b1b36d66ef4325a1f88fb1a1263b45, role_id=111e800ddb0a411c80664cd51ce13fc6)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:46.809010 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523124c2-2996-4662-bc38-4c5ccfb379e3 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:46.820801 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:46.829400 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 204/413] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:46.830072 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdab443-f151-4709-8a20-c293ceaf473f None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:46.892324 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 210/414] 10.222.0.46 () {58 vars in 1442 bytes} [Tue Aug 2 08:46:46 2022] PUT /identity/v3/projects/f8bf83f1361247688208d846845c9d82/groups/d2b1b36d66ef4325a1f88fb1a1263b45/roles/111e800ddb0a411c80664cd51ce13fc6 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:46:46.900094 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523124c2-2996-4662-bc38-4c5ccfb379e3 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:46.966401 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdab443-f151-4709-8a20-c293ceaf473f None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:47.042200 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e773538-1873-4e48-b588-209aaf23c83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:47.042869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e773538-1873-4e48-b588-209aaf23c83d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:47.043151 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e773538-1873-4e48-b588-209aaf23c83d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:47.043425 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e773538-1873-4e48-b588-209aaf23c83d None admin] PATH_INFO: `/v3/groups/723bcb65539446dcba0121b165a90e9c` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:47.045509 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e773538-1873-4e48-b588-209aaf23c83d None admin] RBAC: Authorizing `identity:get_group(group_id=723bcb65539446dcba0121b165a90e9c)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:47.047153 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e773538-1873-4e48-b588-209aaf23c83d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:47.049186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 205/415] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 2 08:46:46 2022] GET /identity/v3/groups/723bcb65539446dcba0121b165a90e9c => generated 229 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:47.055537 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e773538-1873-4e48-b588-209aaf23c83d None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:47.110971 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523124c2-2996-4662-bc38-4c5ccfb379e3 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:47.144300 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:47.158508 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e596c-4f8d-45f4-befb-db9867f70cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:47.164053 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e596c-4f8d-45f4-befb-db9867f70cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:47.164053 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e596c-4f8d-45f4-befb-db9867f70cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:47.164053 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e596c-4f8d-45f4-befb-db9867f70cf6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:47.173389 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0e596c-4f8d-45f4-befb-db9867f70cf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:47.182134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-3e0e596c-4f8d-45f4-befb-db9867f70cf6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:47.182134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:47.182134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:47.182134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:47.182134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:47.182134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:47.182134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:47.182134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:47.182134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:47.182134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:47.182134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:47.182134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:47.182134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:47.182134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:47.182134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:47.182134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:47.183044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:47.183044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:47.183044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:47.183044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:47.183044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:47.183044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:47.183044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:47.183044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:47.183044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:47.183044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:47.183044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:47.183044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:47.183044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:47.183044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:47.183044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:47.183044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:47.183878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:47.183878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:47.183878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:47.183878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:47.183878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:47.183878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:47.183878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:47.183878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:47.183878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:47.183878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:47.183878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:47.183878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:47.184628 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 211/416] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:47.189752 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e773538-1873-4e48-b588-209aaf23c83d None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:47.189910 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e596c-4f8d-45f4-befb-db9867f70cf6 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:47.227508 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560b659-0f76-454c-800c-b797840f05c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:47.227508 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560b659-0f76-454c-800c-b797840f05c8 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:47.227508 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560b659-0f76-454c-800c-b797840f05c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:47.228013 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560b659-0f76-454c-800c-b797840f05c8 None admin] PATH_INFO: `/v3/projects/2a92b68d350c4f7998c221772fb56b4a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:47.229293 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560b659-0f76-454c-800c-b797840f05c8 None admin] RBAC: Authorizing `identity:get_project(project_id=2a92b68d350c4f7998c221772fb56b4a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:47.230839 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560b659-0f76-454c-800c-b797840f05c8 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:47.238395 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 206/417] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:47 2022] GET /identity/v3/projects/2a92b68d350c4f7998c221772fb56b4a => generated 344 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:47.244030 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560b659-0f76-454c-800c-b797840f05c8 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:47.250546 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:47.341703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e596c-4f8d-45f4-befb-db9867f70cf6 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:47.388633 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560b659-0f76-454c-800c-b797840f05c8 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:47.414768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35549c8a-1630-4e12-85c6-c0d8fe1cae85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:47.415405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35549c8a-1630-4e12-85c6-c0d8fe1cae85 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:47.415688 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35549c8a-1630-4e12-85c6-c0d8fe1cae85 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:47.415964 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35549c8a-1630-4e12-85c6-c0d8fe1cae85 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:47.416688 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35549c8a-1630-4e12-85c6-c0d8fe1cae85 None admin] RBAC: Adding query filter params (name=member) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:47.416994 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35549c8a-1630-4e12-85c6-c0d8fe1cae85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:47.418410 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35549c8a-1630-4e12-85c6-c0d8fe1cae85 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:47.438688 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 212/418] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:47.442520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:47.451270 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35549c8a-1630-4e12-85c6-c0d8fe1cae85 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:47.463567 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78f465-0027-4127-8190-12f362ad814a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:47.464319 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78f465-0027-4127-8190-12f362ad814a None admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:47.464607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78f465-0027-4127-8190-12f362ad814a None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:47.464886 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78f465-0027-4127-8190-12f362ad814a None admin] PATH_INFO: `/v3/projects/2a92b68d350c4f7998c221772fb56b4a/groups/723bcb65539446dcba0121b165a90e9c/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:47.469896 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b78f465-0027-4127-8190-12f362ad814a None admin] RBAC: Authorizing `identity:create_grant(project_id=2a92b68d350c4f7998c221772fb56b4a, group_id=723bcb65539446dcba0121b165a90e9c, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:47.471374 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b78f465-0027-4127-8190-12f362ad814a None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:47.558960 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35549c8a-1630-4e12-85c6-c0d8fe1cae85 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:47.574372 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:47.577485 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 207/419] 10.222.0.46 () {58 vars in 1442 bytes} [Tue Aug 2 08:46:47 2022] PUT /identity/v3/projects/2a92b68d350c4f7998c221772fb56b4a/groups/723bcb65539446dcba0121b165a90e9c/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 08:46:47.604138 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188af15c-e116-45b5-84d9-e517daaf11fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:47.604828 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188af15c-e116-45b5-84d9-e517daaf11fb None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:47.605118 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188af15c-e116-45b5-84d9-e517daaf11fb None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:47.605396 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188af15c-e116-45b5-84d9-e517daaf11fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:47.696845 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188af15c-e116-45b5-84d9-e517daaf11fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:47.698591 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188af15c-e116-45b5-84d9-e517daaf11fb None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:47.838141 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 213/420] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:46:47.843526 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78f465-0027-4127-8190-12f362ad814a None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:47.982893 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78f465-0027-4127-8190-12f362ad814a None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:48.018806 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc737135-d64a-4822-a3cb-5ea26e594ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:48.019456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc737135-d64a-4822-a3cb-5ea26e594ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:48.019743 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc737135-d64a-4822-a3cb-5ea26e594ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:48.020021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc737135-d64a-4822-a3cb-5ea26e594ed3 None admin] PATH_INFO: `/v3/projects/592a7a9f7334413584fa4b0d080c1f29` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:48.021183 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc737135-d64a-4822-a3cb-5ea26e594ed3 None admin] RBAC: Authorizing `identity:get_project(project_id=592a7a9f7334413584fa4b0d080c1f29)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:48.022698 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc737135-d64a-4822-a3cb-5ea26e594ed3 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:48.024744 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 208/421] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:47 2022] GET /identity/v3/projects/592a7a9f7334413584fa4b0d080c1f29 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:48.030077 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188af15c-e116-45b5-84d9-e517daaf11fb None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:48.047200 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:48.061702 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e555eb-d126-4b75-8735-ac86e29b2681 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:48.069878 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e555eb-d126-4b75-8735-ac86e29b2681 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:48.069878 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e555eb-d126-4b75-8735-ac86e29b2681 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:48.069878 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 209/422] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:48.069878 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf5c5e-6522-4698-9b1e-2fe1dbab4c1c None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:48.069878 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf5c5e-6522-4698-9b1e-2fe1dbab4c1c None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:48.069878 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf5c5e-6522-4698-9b1e-2fe1dbab4c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:48.092421 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-6ecf5c5e-6522-4698-9b1e-2fe1dbab4c1c None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:48.093620 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-6ecf5c5e-6522-4698-9b1e-2fe1dbab4c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:48.153507 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:48.168848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 210/423] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:46:48.216016 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf5c5e-6522-4698-9b1e-2fe1dbab4c1c None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:48.234113 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188af15c-e116-45b5-84d9-e517daaf11fb None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:48.297829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3eb4c-cf86-42be-8a8c-93a726303ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:48.298391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3eb4c-cf86-42be-8a8c-93a726303ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:48.298882 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3eb4c-cf86-42be-8a8c-93a726303ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:48.298984 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3eb4c-cf86-42be-8a8c-93a726303ca7 None admin] PATH_INFO: `/v3/groups/d2b1b36d66ef4325a1f88fb1a1263b45` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:48.305093 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3eb4c-cf86-42be-8a8c-93a726303ca7 None admin] RBAC: Authorizing `identity:get_group(group_id=d2b1b36d66ef4325a1f88fb1a1263b45)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:48.306614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3eb4c-cf86-42be-8a8c-93a726303ca7 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:48.308769 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 214/424] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 2 08:46:48 2022] GET /identity/v3/groups/d2b1b36d66ef4325a1f88fb1a1263b45 => generated 226 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:48.314277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3eb4c-cf86-42be-8a8c-93a726303ca7 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:48.332104 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf5c5e-6522-4698-9b1e-2fe1dbab4c1c None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:48.370090 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a9061-74fd-4577-8a80-2b76568e4600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:48.371088 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a9061-74fd-4577-8a80-2b76568e4600 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:48.371809 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a9061-74fd-4577-8a80-2b76568e4600 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:48.372279 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a9061-74fd-4577-8a80-2b76568e4600 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:48.384332 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3a9061-74fd-4577-8a80-2b76568e4600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:48.390380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-3e3a9061-74fd-4577-8a80-2b76568e4600 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:48.390380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:48.390380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:48.390380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:48.390380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:48.390380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:48.390380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:48.390380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:48.390380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:48.390380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:48.390380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:48.390380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:48.390380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:48.390380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:48.390380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:48.390380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:48.391734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:48.391734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:48.391734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:48.391734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:48.391734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:48.391734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:48.391734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:48.391734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:48.391734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:48.391734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:48.391734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:48.391734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:48.391734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:48.391734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:48.391734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:48.391734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:48.392562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:48.392562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:48.392562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:48.392562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:48.392562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:48.392562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:48.392562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:48.392562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:48.392562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:48.392562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:48.392562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:48.392562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:48.394747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 211/425] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 2 08:46:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:46:48.396231 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b8710-33d9-45d9-ae42-cea61c69d1a6 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:48.396662 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b8710-33d9-45d9-ae42-cea61c69d1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:48.397130 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6b8710-33d9-45d9-ae42-cea61c69d1a6 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:48.399367 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 212/426] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:48.401220 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b8710-33d9-45d9-ae42-cea61c69d1a6 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:48.461648 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:48.476753 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3eb4c-cf86-42be-8a8c-93a726303ca7 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:48.482292 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:48.517484 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6b8710-33d9-45d9-ae42-cea61c69d1a6 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:48.520686 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cda499-731b-4d60-b00c-5bbf93a4fd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:48.521523 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cda499-731b-4d60-b00c-5bbf93a4fd50 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:48.521988 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cda499-731b-4d60-b00c-5bbf93a4fd50 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:48.523031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cda499-731b-4d60-b00c-5bbf93a4fd50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:48.523980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cda499-731b-4d60-b00c-5bbf93a4fd50 None admin] RBAC: Adding query filter params (group.id=d2b1b36d66ef4325a1f88fb1a1263b45, role.id=68fe920c9be84b3aa1d35bdce554d96a, scope.project.id=592a7a9f7334413584fa4b0d080c1f29) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:48.524435 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cda499-731b-4d60-b00c-5bbf93a4fd50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:48.526081 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cda499-731b-4d60-b00c-5bbf93a4fd50 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:48.534563 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 215/427] 10.222.0.46 () {56 vars in 1376 bytes} [Tue Aug 2 08:46:48 2022] GET /identity/v3/role_assignments?group.id=d2b1b36d66ef4325a1f88fb1a1263b45&scope.project.id=592a7a9f7334413584fa4b0d080c1f29&role.id=68fe920c9be84b3aa1d35bdce554d96a => generated 611 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:48.536035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a361c4f-f1c5-4b54-a663-4fa7c61d8bee None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:48.536472 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a361c4f-f1c5-4b54-a663-4fa7c61d8bee None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:48.536873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a361c4f-f1c5-4b54-a663-4fa7c61d8bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:48.559717 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-6a361c4f-f1c5-4b54-a663-4fa7c61d8bee None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:48.561640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6a361c4f-f1c5-4b54-a663-4fa7c61d8bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:48.596895 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52002f-db42-4723-9c63-19cfda37800c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:48.608894 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52002f-db42-4723-9c63-19cfda37800c None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:48.608894 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52002f-db42-4723-9c63-19cfda37800c None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:48.608894 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52002f-db42-4723-9c63-19cfda37800c None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:48.608894 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52002f-db42-4723-9c63-19cfda37800c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:48.608894 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52002f-db42-4723-9c63-19cfda37800c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:48.608894 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52002f-db42-4723-9c63-19cfda37800c None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:48.624725 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 213/428] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:46:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:48.630318 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52002f-db42-4723-9c63-19cfda37800c None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:48.666241 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 216/429] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:48.695719 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a361c4f-f1c5-4b54-a663-4fa7c61d8bee None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:48.822944 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:48.827972 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52002f-db42-4723-9c63-19cfda37800c None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:48.839472 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:48.849468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a361c4f-f1c5-4b54-a663-4fa7c61d8bee None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:48.865102 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb65b5-28ee-4e9e-b92f-305ac14e8ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:48.866001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb65b5-28ee-4e9e-b92f-305ac14e8ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:48.867218 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb65b5-28ee-4e9e-b92f-305ac14e8ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:48.867647 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb65b5-28ee-4e9e-b92f-305ac14e8ab3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:48.892862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda6313-6fa0-4604-a1bc-ddc8477b67dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:48.893681 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda6313-6fa0-4604-a1bc-ddc8477b67dd None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:48.894136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda6313-6fa0-4604-a1bc-ddc8477b67dd None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:48.894662 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda6313-6fa0-4604-a1bc-ddc8477b67dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:48.908384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda6313-6fa0-4604-a1bc-ddc8477b67dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:48.919035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-ddda6313-6fa0-4604-a1bc-ddc8477b67dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:48.919035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:48.919035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:48.919035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:48.919035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:48.919035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:48.919035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:48.919035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:48.919035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:48.919035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:48.919035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:48.919035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:48.919035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:48.919035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:48.919035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:48.919035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:48.920187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:48.920187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:48.920187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:48.920187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:48.920187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:48.920187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:48.920187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:48.920187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:48.920187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:48.920187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:48.920187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:48.920187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:48.920187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:48.920187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:48.920187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:48.920187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:48.921104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:48.921104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:48.921104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:48.921104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:48.921104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:48.921104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:48.921104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:48.921104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:48.921104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:48.921104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:48.921104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 08:46:48.921104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:48.921740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 217/430] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:48.928215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda6313-6fa0-4604-a1bc-ddc8477b67dd None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:48.968928 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb65b5-28ee-4e9e-b92f-305ac14e8ab3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:48.971131 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb65b5-28ee-4e9e-b92f-305ac14e8ab3 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:49.058509 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda6313-6fa0-4604-a1bc-ddc8477b67dd None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:49.091806 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 214/431] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:46:49.096930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebcf13-499b-448c-81a6-1e1b411bd4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:49.097703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebcf13-499b-448c-81a6-1e1b411bd4e0 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:49.097936 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb65b5-28ee-4e9e-b92f-305ac14e8ab3 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:49.098098 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebcf13-499b-448c-81a6-1e1b411bd4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:49.098552 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebcf13-499b-448c-81a6-1e1b411bd4e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:49.099358 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ebcf13-499b-448c-81a6-1e1b411bd4e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:49.099732 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ebcf13-499b-448c-81a6-1e1b411bd4e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:49.101976 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ebcf13-499b-448c-81a6-1e1b411bd4e0 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:49.103652 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:49.111817 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:49.115707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 218/432] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:46:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:49.120101 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebcf13-499b-448c-81a6-1e1b411bd4e0 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:49.216085 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb65b5-28ee-4e9e-b92f-305ac14e8ab3 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:49.251765 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebcf13-499b-448c-81a6-1e1b411bd4e0 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:49.253588 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6813a3-1cd3-4ebd-ab71-a26ff5fb364f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:49.254276 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6813a3-1cd3-4ebd-ab71-a26ff5fb364f None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:49.254561 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6813a3-1cd3-4ebd-ab71-a26ff5fb364f None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:49.254905 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6813a3-1cd3-4ebd-ab71-a26ff5fb364f None admin] PATH_INFO: `/v3/projects/592a7a9f7334413584fa4b0d080c1f29` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:49.256146 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6813a3-1cd3-4ebd-ab71-a26ff5fb364f None admin] RBAC: Authorizing `identity:get_project(project_id=592a7a9f7334413584fa4b0d080c1f29)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:49.258965 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6813a3-1cd3-4ebd-ab71-a26ff5fb364f None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:49.265294 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 215/433] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:49 2022] GET /identity/v3/projects/592a7a9f7334413584fa4b0d080c1f29 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:49.266202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6813a3-1cd3-4ebd-ab71-a26ff5fb364f None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:49.298293 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098b2f1-a5ea-4eac-a140-1a4d3f0a9c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:49.299042 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098b2f1-a5ea-4eac-a140-1a4d3f0a9c38 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:49.299332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098b2f1-a5ea-4eac-a140-1a4d3f0a9c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:49.299610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098b2f1-a5ea-4eac-a140-1a4d3f0a9c38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:49.369369 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:49.372896 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:49.379082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098b2f1-a5ea-4eac-a140-1a4d3f0a9c38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:49.380945 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3098b2f1-a5ea-4eac-a140-1a4d3f0a9c38 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:49.403813 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6813a3-1cd3-4ebd-ab71-a26ff5fb364f None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:49.449082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6f6ec-fe41-4b75-a385-a2dbece9332d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:49.450223 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6f6ec-fe41-4b75-a385-a2dbece9332d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:49.450512 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6f6ec-fe41-4b75-a385-a2dbece9332d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:49.451128 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6f6ec-fe41-4b75-a385-a2dbece9332d None admin] PATH_INFO: `/v3/groups/d2b1b36d66ef4325a1f88fb1a1263b45` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:49.453976 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6f6ec-fe41-4b75-a385-a2dbece9332d None admin] RBAC: Authorizing `identity:get_group(group_id=d2b1b36d66ef4325a1f88fb1a1263b45)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:49.455689 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6f6ec-fe41-4b75-a385-a2dbece9332d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:49.458142 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 216/434] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 2 08:46:49 2022] GET /identity/v3/groups/d2b1b36d66ef4325a1f88fb1a1263b45 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:49.459438 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356659e4-3c2f-4a3d-9e58-467fc7b93cc6 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:49.459729 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356659e4-3c2f-4a3d-9e58-467fc7b93cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:49.460041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356659e4-3c2f-4a3d-9e58-467fc7b93cc6 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:49.462742 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 217/435] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:49.466205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356659e4-3c2f-4a3d-9e58-467fc7b93cc6 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:49.512571 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 219/436] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 08:46:49.518330 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d8e48-d890-4adb-b3c3-e664ef1ae9a8 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:49.518667 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d8e48-d890-4adb-b3c3-e664ef1ae9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:49.518875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d8e48-d890-4adb-b3c3-e664ef1ae9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:49.539425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9a6d8e48-d890-4adb-b3c3-e664ef1ae9a8 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:49.540913 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9a6d8e48-d890-4adb-b3c3-e664ef1ae9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:49.577644 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356659e4-3c2f-4a3d-9e58-467fc7b93cc6 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:49.614511 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 220/437] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:49.615432 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d8e48-d890-4adb-b3c3-e664ef1ae9a8 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:49.623366 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5103b5d-90f0-4910-a006-3ef8d49f1eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:49.624673 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5103b5d-90f0-4910-a006-3ef8d49f1eac None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:49.625181 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5103b5d-90f0-4910-a006-3ef8d49f1eac None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:49.625671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5103b5d-90f0-4910-a006-3ef8d49f1eac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:49.627088 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5103b5d-90f0-4910-a006-3ef8d49f1eac None admin] RBAC: Adding query filter params (group.id=d2b1b36d66ef4325a1f88fb1a1263b45, role.id=111e800ddb0a411c80664cd51ce13fc6, scope.project.id=592a7a9f7334413584fa4b0d080c1f29) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:49.627663 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5103b5d-90f0-4910-a006-3ef8d49f1eac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:49.630083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5103b5d-90f0-4910-a006-3ef8d49f1eac None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:49.638011 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 218/438] 10.222.0.46 () {56 vars in 1376 bytes} [Tue Aug 2 08:46:49 2022] GET /identity/v3/role_assignments?group.id=d2b1b36d66ef4325a1f88fb1a1263b45&scope.project.id=592a7a9f7334413584fa4b0d080c1f29&role.id=111e800ddb0a411c80664cd51ce13fc6 => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:49.639149 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5103b5d-90f0-4910-a006-3ef8d49f1eac None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:49.650475 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:49.666163 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:49.750218 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d8e48-d890-4adb-b3c3-e664ef1ae9a8 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:49.777105 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5103b5d-90f0-4910-a006-3ef8d49f1eac None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:49.804334 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a1eabc-b448-4662-844b-883cbb40532a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:49.805340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a1eabc-b448-4662-844b-883cbb40532a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:49.805838 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a1eabc-b448-4662-844b-883cbb40532a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:49.806281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a1eabc-b448-4662-844b-883cbb40532a None admin] PATH_INFO: `/v3/projects/f8bf83f1361247688208d846845c9d82` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:49.807817 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a1eabc-b448-4662-844b-883cbb40532a None admin] RBAC: Authorizing `identity:get_project(project_id=f8bf83f1361247688208d846845c9d82)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:49.809781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a1eabc-b448-4662-844b-883cbb40532a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:49.815338 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 221/439] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:49 2022] GET /identity/v3/projects/f8bf83f1361247688208d846845c9d82 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:49.816198 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a1eabc-b448-4662-844b-883cbb40532a None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:49.829442 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080554f6-f8f0-4bf4-80bf-1aee609c87eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:49.830732 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080554f6-f8f0-4bf4-80bf-1aee609c87eb None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:49.831251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080554f6-f8f0-4bf4-80bf-1aee609c87eb None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:49.831990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080554f6-f8f0-4bf4-80bf-1aee609c87eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:49.844118 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080554f6-f8f0-4bf4-80bf-1aee609c87eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:49.849406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-080554f6-f8f0-4bf4-80bf-1aee609c87eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:49.849406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:49.849406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:49.849406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:49.849406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:49.849406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:49.849406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:49.849406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:49.849406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:49.849406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:49.849406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:49.849406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:49.849406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:49.849406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:49.849406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:49.849406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:49.850361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:49.850361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:49.850361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:49.850361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:49.850361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:49.850361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:49.850361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:49.850361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:49.850361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:49.850361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:49.850361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:49.850361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:49.850361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:49.850361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:49.850361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:49.850361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:49.851727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:49.851727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:49.851727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:49.851727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:49.851727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:49.851727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:49.851727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:49.851727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:49.851727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:49.851727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:49.851727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:49.851727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:49.854435 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 219/440] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 2 08:46:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:49.860317 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080554f6-f8f0-4bf4-80bf-1aee609c87eb None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:49.938785 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:49.947987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a1eabc-b448-4662-844b-883cbb40532a None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:49.966061 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:49.996017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43756c4e-c5e5-44fa-94f4-3dce197ed3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:49.996731 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43756c4e-c5e5-44fa-94f4-3dce197ed3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:49.997021 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43756c4e-c5e5-44fa-94f4-3dce197ed3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:49.997283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43756c4e-c5e5-44fa-94f4-3dce197ed3a4 None admin] PATH_INFO: `/v3/groups/d2b1b36d66ef4325a1f88fb1a1263b45` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:49.999605 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43756c4e-c5e5-44fa-94f4-3dce197ed3a4 None admin] RBAC: Authorizing `identity:get_group(group_id=d2b1b36d66ef4325a1f88fb1a1263b45)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:50.000601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43756c4e-c5e5-44fa-94f4-3dce197ed3a4 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:50.002470 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 222/441] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 2 08:46:49 2022] GET /identity/v3/groups/d2b1b36d66ef4325a1f88fb1a1263b45 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:50.009685 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43756c4e-c5e5-44fa-94f4-3dce197ed3a4 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:50.011709 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080554f6-f8f0-4bf4-80bf-1aee609c87eb None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:50.059371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72f9e8-6c51-4c6d-883f-f3c8c314cbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:50.059994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72f9e8-6c51-4c6d-883f-f3c8c314cbe3 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:50.060276 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72f9e8-6c51-4c6d-883f-f3c8c314cbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:50.060557 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a72f9e8-6c51-4c6d-883f-f3c8c314cbe3 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:50.061279 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a72f9e8-6c51-4c6d-883f-f3c8c314cbe3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:50.061645 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a72f9e8-6c51-4c6d-883f-f3c8c314cbe3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:50.063122 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a72f9e8-6c51-4c6d-883f-f3c8c314cbe3 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:50.075358 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 220/442] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:46:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:50.084282 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72f9e8-6c51-4c6d-883f-f3c8c314cbe3 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:50.135396 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43756c4e-c5e5-44fa-94f4-3dce197ed3a4 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:50.169392 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d9282-4c5c-41ff-90d9-cd2179bfc156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:50.170042 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d9282-4c5c-41ff-90d9-cd2179bfc156 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:50.170324 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d9282-4c5c-41ff-90d9-cd2179bfc156 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:50.170609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d9282-4c5c-41ff-90d9-cd2179bfc156 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:50.171392 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d9282-4c5c-41ff-90d9-cd2179bfc156 None admin] RBAC: Adding query filter params (group.id=d2b1b36d66ef4325a1f88fb1a1263b45, role.id=68fe920c9be84b3aa1d35bdce554d96a, scope.project.id=f8bf83f1361247688208d846845c9d82) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:50.171703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d9282-4c5c-41ff-90d9-cd2179bfc156 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:50.173131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d9282-4c5c-41ff-90d9-cd2179bfc156 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:50.178845 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 223/443] 10.222.0.46 () {56 vars in 1376 bytes} [Tue Aug 2 08:46:50 2022] GET /identity/v3/role_assignments?group.id=d2b1b36d66ef4325a1f88fb1a1263b45&scope.project.id=f8bf83f1361247688208d846845c9d82&role.id=68fe920c9be84b3aa1d35bdce554d96a => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:50.179981 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb387594-7989-43f5-a8dc-32d1fcfec1e4 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:50.180265 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb387594-7989-43f5-a8dc-32d1fcfec1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:50.180542 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb387594-7989-43f5-a8dc-32d1fcfec1e4 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:50.181929 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 224/444] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:50.194077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a72f9e8-6c51-4c6d-883f-f3c8c314cbe3 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:50.196691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2705fa-f15f-42c4-9aa0-3936b1ae92fc None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:50.196988 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2705fa-f15f-42c4-9aa0-3936b1ae92fc None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:50.198181 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2705fa-f15f-42c4-9aa0-3936b1ae92fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:50.218102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-fd2705fa-f15f-42c4-9aa0-3936b1ae92fc None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:50.219329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-fd2705fa-f15f-42c4-9aa0-3936b1ae92fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:50.231014 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:50.236737 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b99b96-72d0-46d7-bdaa-337a1b2610c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:50.237432 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b99b96-72d0-46d7-bdaa-337a1b2610c4 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:50.237648 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b99b96-72d0-46d7-bdaa-337a1b2610c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:50.237949 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b99b96-72d0-46d7-bdaa-337a1b2610c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:50.243661 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:50.302051 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 225/445] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:50.324476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2705fa-f15f-42c4-9aa0-3936b1ae92fc None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:50.333171 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b99b96-72d0-46d7-bdaa-337a1b2610c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:50.335660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b99b96-72d0-46d7-bdaa-337a1b2610c4 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:50.440696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2705fa-f15f-42c4-9aa0-3936b1ae92fc None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:50.464992 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 221/446] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:46:50.470584 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b99b96-72d0-46d7-bdaa-337a1b2610c4 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:50.487343 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcb2ae-1f15-4ea2-896b-7279644c1349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:50.488010 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcb2ae-1f15-4ea2-896b-7279644c1349 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:50.488289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcb2ae-1f15-4ea2-896b-7279644c1349 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:50.488569 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcb2ae-1f15-4ea2-896b-7279644c1349 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:50.501764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dcb2ae-1f15-4ea2-896b-7279644c1349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:50.506826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-91dcb2ae-1f15-4ea2-896b-7279644c1349 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:46:50.506826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:50.506826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:50.506826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:50.506826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:50.506826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:50.506826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:50.506826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:50.506826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:50.506826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:50.506826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:50.506826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:50.506826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:50.506826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:50.506826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:50.506826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:50.507741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:50.507741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:50.507741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:50.507741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:50.507741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:50.507741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:50.507741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:50.507741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:50.507741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:50.507741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:50.507741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:50.507741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:50.507741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:50.507741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:50.507741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:50.507741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:50.508691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:50.508691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:50.508691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:50.508691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:50.508691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:50.508691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:50.508691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:50.508691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:50.508691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:50.508691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:50.508691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:46:50.508691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:50.508691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 226/447] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:46:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:46:50.510714 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:50.514750 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcb2ae-1f15-4ea2-896b-7279644c1349 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:50.536815 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:50.584584 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b99b96-72d0-46d7-bdaa-337a1b2610c4 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:50.618361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be434a92-d4f2-46ab-9415-6a5c6f9ccc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:50.618988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be434a92-d4f2-46ab-9415-6a5c6f9ccc5d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:50.619266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be434a92-d4f2-46ab-9415-6a5c6f9ccc5d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:50.619537 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be434a92-d4f2-46ab-9415-6a5c6f9ccc5d None admin] PATH_INFO: `/v3/projects/f8bf83f1361247688208d846845c9d82` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:50.620664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be434a92-d4f2-46ab-9415-6a5c6f9ccc5d None admin] RBAC: Authorizing `identity:get_project(project_id=f8bf83f1361247688208d846845c9d82)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:50.622132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be434a92-d4f2-46ab-9415-6a5c6f9ccc5d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:50.622872 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcb2ae-1f15-4ea2-896b-7279644c1349 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:50.623768 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 222/448] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:50 2022] GET /identity/v3/projects/f8bf83f1361247688208d846845c9d82 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:50.629362 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be434a92-d4f2-46ab-9415-6a5c6f9ccc5d None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:50.660278 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c677d-89d7-4783-a4a8-8af843762194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:50.660897 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c677d-89d7-4783-a4a8-8af843762194 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:50.661173 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c677d-89d7-4783-a4a8-8af843762194 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:50.661449 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c677d-89d7-4783-a4a8-8af843762194 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:50.662192 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c677d-89d7-4783-a4a8-8af843762194 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:50.662495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c677d-89d7-4783-a4a8-8af843762194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:50.664455 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c677d-89d7-4783-a4a8-8af843762194 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:50.672547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 227/449] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:50.678026 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c677d-89d7-4783-a4a8-8af843762194 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:50.735834 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:50.738550 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be434a92-d4f2-46ab-9415-6a5c6f9ccc5d None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:50.756005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:50.772008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7b801-d02c-41fa-9f93-44db64e114ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:50.773210 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7b801-d02c-41fa-9f93-44db64e114ae None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:50.773489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7b801-d02c-41fa-9f93-44db64e114ae None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:50.773761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7b801-d02c-41fa-9f93-44db64e114ae None admin] PATH_INFO: `/v3/groups/d2b1b36d66ef4325a1f88fb1a1263b45` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:50.774925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7b801-d02c-41fa-9f93-44db64e114ae None admin] RBAC: Authorizing `identity:get_group(group_id=d2b1b36d66ef4325a1f88fb1a1263b45)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:50.776339 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7b801-d02c-41fa-9f93-44db64e114ae None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:50.777668 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c677d-89d7-4783-a4a8-8af843762194 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:50.777971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 223/450] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 2 08:46:50 2022] GET /identity/v3/groups/d2b1b36d66ef4325a1f88fb1a1263b45 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:50.783668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7b801-d02c-41fa-9f93-44db64e114ae None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:50.811417 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281a8c0-46d4-42a0-b124-0015e84a2c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:50.812038 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281a8c0-46d4-42a0-b124-0015e84a2c42 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:50.812320 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281a8c0-46d4-42a0-b124-0015e84a2c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:50.812596 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281a8c0-46d4-42a0-b124-0015e84a2c42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:50.881102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281a8c0-46d4-42a0-b124-0015e84a2c42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:50.882480 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7b801-d02c-41fa-9f93-44db64e114ae None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:50.883124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281a8c0-46d4-42a0-b124-0015e84a2c42 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:50.916328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee29e4e-8242-4f15-bfe4-f12df5e33a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:50.916937 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee29e4e-8242-4f15-bfe4-f12df5e33a8d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:50.917225 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee29e4e-8242-4f15-bfe4-f12df5e33a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:50.917506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee29e4e-8242-4f15-bfe4-f12df5e33a8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:50.918320 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee29e4e-8242-4f15-bfe4-f12df5e33a8d None admin] RBAC: Adding query filter params (group.id=d2b1b36d66ef4325a1f88fb1a1263b45, role.id=111e800ddb0a411c80664cd51ce13fc6, scope.project.id=f8bf83f1361247688208d846845c9d82) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:50.918636 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee29e4e-8242-4f15-bfe4-f12df5e33a8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:50.920056 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee29e4e-8242-4f15-bfe4-f12df5e33a8d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:50.927202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 224/451] 10.222.0.46 () {56 vars in 1376 bytes} [Tue Aug 2 08:46:50 2022] GET /identity/v3/role_assignments?group.id=d2b1b36d66ef4325a1f88fb1a1263b45&scope.project.id=f8bf83f1361247688208d846845c9d82&role.id=111e800ddb0a411c80664cd51ce13fc6 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:50.964702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:50.996285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 228/452] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 08:46:51.002425 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee29e4e-8242-4f15-bfe4-f12df5e33a8d None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:51.035964 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:51.108725 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee29e4e-8242-4f15-bfe4-f12df5e33a8d None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:51.144781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc285b5d-8894-494a-8d9b-94fae8dfb528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:51.145429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc285b5d-8894-494a-8d9b-94fae8dfb528 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:51.145712 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc285b5d-8894-494a-8d9b-94fae8dfb528 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:51.146018 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc285b5d-8894-494a-8d9b-94fae8dfb528 None admin] PATH_INFO: `/v3/projects/2a92b68d350c4f7998c221772fb56b4a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:51.147188 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc285b5d-8894-494a-8d9b-94fae8dfb528 None admin] RBAC: Authorizing `identity:get_project(project_id=2a92b68d350c4f7998c221772fb56b4a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:51.149315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc285b5d-8894-494a-8d9b-94fae8dfb528 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:51.151041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 225/453] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:46:51 2022] GET /identity/v3/projects/2a92b68d350c4f7998c221772fb56b4a => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:51.156679 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281a8c0-46d4-42a0-b124-0015e84a2c42 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:51.258230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281a8c0-46d4-42a0-b124-0015e84a2c42 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:51.292462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5dd06-7793-4b4d-aca5-a46bdc99a70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:51.293221 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5dd06-7793-4b4d-aca5-a46bdc99a70f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:51.293506 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5dd06-7793-4b4d-aca5-a46bdc99a70f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:51.293782 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5dd06-7793-4b4d-aca5-a46bdc99a70f None admin] PATH_INFO: `/v3/groups/723bcb65539446dcba0121b165a90e9c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:51.295023 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c5dd06-7793-4b4d-aca5-a46bdc99a70f None admin] RBAC: Authorizing `identity:get_group(group_id=723bcb65539446dcba0121b165a90e9c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:51.296467 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c5dd06-7793-4b4d-aca5-a46bdc99a70f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:51.298088 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 229/454] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 2 08:46:51 2022] GET /identity/v3/groups/723bcb65539446dcba0121b165a90e9c => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:51.304026 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc285b5d-8894-494a-8d9b-94fae8dfb528 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:51.399058 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:51.401855 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc285b5d-8894-494a-8d9b-94fae8dfb528 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:51.434381 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a623dbf4-b8f8-4593-88ff-598b83612e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:51.434982 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623dbf4-b8f8-4593-88ff-598b83612e64 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:51.435266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623dbf4-b8f8-4593-88ff-598b83612e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:51.435544 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a623dbf4-b8f8-4593-88ff-598b83612e64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:51.436322 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623dbf4-b8f8-4593-88ff-598b83612e64 None admin] RBAC: Adding query filter params (group.id=723bcb65539446dcba0121b165a90e9c, role.id=8ce0850e15804f3d811e88cc9841e673, scope.project.id=2a92b68d350c4f7998c221772fb56b4a) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:51.436633 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623dbf4-b8f8-4593-88ff-598b83612e64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:51.438151 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a623dbf4-b8f8-4593-88ff-598b83612e64 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:51.444029 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 226/455] 10.222.0.46 () {56 vars in 1376 bytes} [Tue Aug 2 08:46:51 2022] GET /identity/v3/role_assignments?group.id=723bcb65539446dcba0121b165a90e9c&scope.project.id=2a92b68d350c4f7998c221772fb56b4a&role.id=8ce0850e15804f3d811e88cc9841e673 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:53.905417 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e595d-4e05-4c90-815e-acd70a9b8a2b None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:53.905417 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e595d-4e05-4c90-815e-acd70a9b8a2b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:53.905417 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e595d-4e05-4c90-815e-acd70a9b8a2b None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:53.906361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 230/456] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:53.911921 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2551c7-5c07-42cb-9261-7d150a702837 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:53.912319 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2551c7-5c07-42cb-9261-7d150a702837 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:53.912543 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2551c7-5c07-42cb-9261-7d150a702837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:53.951390 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-4e2551c7-5c07-42cb-9261-7d150a702837 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:53.952607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4e2551c7-5c07-42cb-9261-7d150a702837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:54.050633 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 227/457] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:54.092450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e595d-4e05-4c90-815e-acd70a9b8a2b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:54.105250 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1a258-5015-4f32-a302-2a8d6485ea83 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:54.105580 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1a258-5015-4f32-a302-2a8d6485ea83 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:54.105863 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1a258-5015-4f32-a302-2a8d6485ea83 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:54.109050 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 228/458] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:54.114754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7225c-e9ba-4fe7-8b09-f8cbce227b2d None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:54.114754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7225c-e9ba-4fe7-8b09-f8cbce227b2d None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:54.114754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7225c-e9ba-4fe7-8b09-f8cbce227b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:54.151735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-31b7225c-e9ba-4fe7-8b09-f8cbce227b2d None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:54.152954 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-31b7225c-e9ba-4fe7-8b09-f8cbce227b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:54.185995 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:54.245912 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e595d-4e05-4c90-815e-acd70a9b8a2b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:54.294312 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 229/459] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:46:54.297107 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9ad13-31a4-47a4-ba58-1e30cf1d8343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:54.298054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9ad13-31a4-47a4-ba58-1e30cf1d8343 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:54.298561 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9ad13-31a4-47a4-ba58-1e30cf1d8343 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:54.299027 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9ad13-31a4-47a4-ba58-1e30cf1d8343 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:54.315823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9ad13-31a4-47a4-ba58-1e30cf1d8343 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:54.317578 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9ad13-31a4-47a4-ba58-1e30cf1d8343 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:54.341398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-35a9ad13-31a4-47a4-ba58-1e30cf1d8343 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:46:54.341398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:54.341398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:54.341398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:54.341398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:54.341398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:54.341398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:54.341398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:54.341398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:54.341398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:54.341398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:54.341398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:54.341398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:46:54.341398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:46:54.341398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:46:54.342352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:46:54.342352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:54.342352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:54.342352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:54.342352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:54.342352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:54.342352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:54.342352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:54.342352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:54.342352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:54.342352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:54.342352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:54.342352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:54.342352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:54.342352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:54.342352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:54.342352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:54.343190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:46:54.343190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:46:54.343190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:46:54.343190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:54.345431 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 231/460] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:54.352088 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7225c-e9ba-4fe7-8b09-f8cbce227b2d None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:54.362378 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee0c80-84cc-4ada-8d37-d2e40416cb1c None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:54.362947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee0c80-84cc-4ada-8d37-d2e40416cb1c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:54.363417 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee0c80-84cc-4ada-8d37-d2e40416cb1c None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:54.365243 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 232/461] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:54.371562 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ecdc1-2068-4156-85f3-38b2c1b38147 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:54.372154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ecdc1-2068-4156-85f3-38b2c1b38147 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:54.372657 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822ecdc1-2068-4156-85f3-38b2c1b38147 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:54.374068 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 233/462] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:46:54.375046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc6e65-f040-4000-a4c2-b31d03e17e3b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:54.375328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc6e65-f040-4000-a4c2-b31d03e17e3b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:54.375687 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafc6e65-f040-4000-a4c2-b31d03e17e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:54.388220 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:54.394121 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-dafc6e65-f040-4000-a4c2-b31d03e17e3b None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:54.395379 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-dafc6e65-f040-4000-a4c2-b31d03e17e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:54.465722 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 234/463] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:54.467451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7225c-e9ba-4fe7-8b09-f8cbce227b2d None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:54.468497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ce80b-5612-45a1-a6b7-47bff391ae4c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:54.468900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ce80b-5612-45a1-a6b7-47bff391ae4c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:54.469182 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ce80b-5612-45a1-a6b7-47bff391ae4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:54.493133 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-937ce80b-5612-45a1-a6b7-47bff391ae4c None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:54.494630 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-937ce80b-5612-45a1-a6b7-47bff391ae4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:54.509008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c7e3e-5288-4280-97cc-832b0a290ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:54.509957 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c7e3e-5288-4280-97cc-832b0a290ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:54.510242 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c7e3e-5288-4280-97cc-832b0a290ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:54.510515 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c7e3e-5288-4280-97cc-832b0a290ad6 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:54.511741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c7e3e-5288-4280-97cc-832b0a290ad6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:54.512050 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c7e3e-5288-4280-97cc-832b0a290ad6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:54.513461 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c7e3e-5288-4280-97cc-832b0a290ad6 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:54.523616 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:54.525968 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 230/464] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:46:54.531109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c7e3e-5288-4280-97cc-832b0a290ad6 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:54.573439 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 235/465] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:54.574309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ce80b-5612-45a1-a6b7-47bff391ae4c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:54.628303 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:54.643978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c7e3e-5288-4280-97cc-832b0a290ad6 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:54.680473 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975bca8-7695-4b87-b129-49284506c476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:54.681527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975bca8-7695-4b87-b129-49284506c476 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:54.681811 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975bca8-7695-4b87-b129-49284506c476 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:54.682123 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975bca8-7695-4b87-b129-49284506c476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:54.684462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ce80b-5612-45a1-a6b7-47bff391ae4c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:54.696399 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975bca8-7695-4b87-b129-49284506c476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:54.698222 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d975bca8-7695-4b87-b129-49284506c476 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:54.714302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-d975bca8-7695-4b87-b129-49284506c476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:46:54.714302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:54.714302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:54.714302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:54.714302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:54.714302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:54.714302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:54.714302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:54.714302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:54.714302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:54.714302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:54.714302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:54.714302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:46:54.714302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:46:54.714302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:46:54.715230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:46:54.715230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:54.715230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:54.715230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:54.715230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:54.715230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:54.715230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:54.715230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:54.715230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:54.715230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:54.715230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:54.715230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:54.715230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:54.715230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:54.715230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:54.715230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:54.715230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:54.716140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:46:54.716140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:46:54.716140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:46:54.716140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:54.716140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 231/466] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:54.716436 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975bca8-7695-4b87-b129-49284506c476 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:54.726369 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72678f66-ec34-48c4-8bd9-cc980d0126cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:54.727018 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72678f66-ec34-48c4-8bd9-cc980d0126cd None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:54.727304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72678f66-ec34-48c4-8bd9-cc980d0126cd None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:54.727570 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72678f66-ec34-48c4-8bd9-cc980d0126cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:54.738179 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72678f66-ec34-48c4-8bd9-cc980d0126cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:54.739603 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72678f66-ec34-48c4-8bd9-cc980d0126cd None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:54.754782 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-72678f66-ec34-48c4-8bd9-cc980d0126cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:46:54.754782 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:54.754782 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:54.754782 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:54.754782 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:54.754782 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:54.754782 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:54.754782 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:54.754782 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:54.754782 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:54.754782 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:54.754782 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:54.754782 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:46:54.754782 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:46:54.754782 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:46:54.755612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:46:54.755612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:54.755612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:54.755612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:54.755612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:54.755612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:54.755612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:54.755612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:54.755612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:54.755612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:54.755612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:54.755612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:54.755612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:54.755612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:54.755612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:54.755612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:54.755612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:54.756457 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:46:54.756457 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:46:54.756457 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:46:54.756457 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:54.756457 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 236/467] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:54.756752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72678f66-ec34-48c4-8bd9-cc980d0126cd None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:54.772465 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:54.825169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975bca8-7695-4b87-b129-49284506c476 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:54.861425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72678f66-ec34-48c4-8bd9-cc980d0126cd None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:54.862981 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427662a-92dd-4ba3-95ec-f4a79fe785bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:54.863960 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427662a-92dd-4ba3-95ec-f4a79fe785bf None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:54.864312 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427662a-92dd-4ba3-95ec-f4a79fe785bf None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:54.864590 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427662a-92dd-4ba3-95ec-f4a79fe785bf None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:54.865268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427662a-92dd-4ba3-95ec-f4a79fe785bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:54.867283 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427662a-92dd-4ba3-95ec-f4a79fe785bf None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:54.876783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:54.895097 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d81cd-de8f-4623-8268-59f63c675dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:54.895728 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d81cd-de8f-4623-8268-59f63c675dee None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:54.896007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d81cd-de8f-4623-8268-59f63c675dee None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:54.896282 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d81cd-de8f-4623-8268-59f63c675dee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:54.907190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d81cd-de8f-4623-8268-59f63c675dee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:54.908623 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d81cd-de8f-4623-8268-59f63c675dee None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:54.923370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-5d0d81cd-de8f-4623-8268-59f63c675dee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:46:54.923370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:54.923370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:54.923370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:54.923370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:54.923370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:54.923370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:54.923370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:54.923370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:54.923370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:54.923370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:54.923370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:54.923370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:46:54.923370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:46:54.923370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:46:54.924251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:46:54.924251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:54.924251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:54.924251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:54.924251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:54.924251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:54.924251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:54.924251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:54.924251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:54.924251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:54.924251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:54.924251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:54.924251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:54.924251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:54.924251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:54.924251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:54.924251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:54.930936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:46:54.930936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:46:54.930936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:46:54.930936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:54.930936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 237/468] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:54.930936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d81cd-de8f-4623-8268-59f63c675dee None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:54.968524 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 232/469] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 2 08:46:54 2022] POST /identity/v3/users => generated 246 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:46:54.969204 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427662a-92dd-4ba3-95ec-f4a79fe785bf None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:55.013855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:55.071593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d81cd-de8f-4623-8268-59f63c675dee None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:55.109666 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427662a-92dd-4ba3-95ec-f4a79fe785bf None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:55.126053 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87df38b-7bd1-496c-90a0-9dc495374793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:55.127178 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87df38b-7bd1-496c-90a0-9dc495374793 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:55.127471 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87df38b-7bd1-496c-90a0-9dc495374793 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:55.127752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87df38b-7bd1-496c-90a0-9dc495374793 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:55.128736 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87df38b-7bd1-496c-90a0-9dc495374793 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:55.129318 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87df38b-7bd1-496c-90a0-9dc495374793 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:55.130718 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87df38b-7bd1-496c-90a0-9dc495374793 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:55.154020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 238/470] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:55.154020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87df38b-7bd1-496c-90a0-9dc495374793 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:55.154296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0616e9-2100-4c52-a609-a6dcddd9e046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:55.154296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0616e9-2100-4c52-a609-a6dcddd9e046 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:55.154296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0616e9-2100-4c52-a609-a6dcddd9e046 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:55.154296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0616e9-2100-4c52-a609-a6dcddd9e046 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:55.154890 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0616e9-2100-4c52-a609-a6dcddd9e046 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:55.156944 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0616e9-2100-4c52-a609-a6dcddd9e046 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:55.156944 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0616e9-2100-4c52-a609-a6dcddd9e046 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:55.170383 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 233/471] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:55.171044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0616e9-2100-4c52-a609-a6dcddd9e046 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:55.251721 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:55.273011 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87df38b-7bd1-496c-90a0-9dc495374793 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:55.297320 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:55.309187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f3aae-50be-4a34-800c-bc39fb5c0de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:55.310576 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f3aae-50be-4a34-800c-bc39fb5c0de6 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:55.311093 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f3aae-50be-4a34-800c-bc39fb5c0de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:55.311564 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f3aae-50be-4a34-800c-bc39fb5c0de6 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:55.312426 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0616e9-2100-4c52-a609-a6dcddd9e046 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:55.312679 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f3aae-50be-4a34-800c-bc39fb5c0de6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:55.313170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f3aae-50be-4a34-800c-bc39fb5c0de6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:55.314840 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f3aae-50be-4a34-800c-bc39fb5c0de6 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:55.327855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 239/472] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:55.329016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f3aae-50be-4a34-800c-bc39fb5c0de6 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:55.356268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8c7e5-2025-4fc5-a3a2-292339fec09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:55.356950 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8c7e5-2025-4fc5-a3a2-292339fec09d None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:55.357237 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8c7e5-2025-4fc5-a3a2-292339fec09d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:55.357513 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8c7e5-2025-4fc5-a3a2-292339fec09d None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:55.358567 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8c7e5-2025-4fc5-a3a2-292339fec09d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:55.360226 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8c7e5-2025-4fc5-a3a2-292339fec09d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:55.439497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f3aae-50be-4a34-800c-bc39fb5c0de6 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:55.450001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 234/473] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 2 08:46:55 2022] POST /identity/v3/users => generated 248 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:46:55.450001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8c7e5-2025-4fc5-a3a2-292339fec09d None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:55.490009 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82256539-e447-489e-a6e2-81cafbc53b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:55.491712 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82256539-e447-489e-a6e2-81cafbc53b13 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:55.492652 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82256539-e447-489e-a6e2-81cafbc53b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:55.493256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82256539-e447-489e-a6e2-81cafbc53b13 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:55.494233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82256539-e447-489e-a6e2-81cafbc53b13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:55.496989 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82256539-e447-489e-a6e2-81cafbc53b13 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:55.582651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8c7e5-2025-4fc5-a3a2-292339fec09d None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:55.593151 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:55.613782 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 240/474] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 2 08:46:55 2022] POST /identity/v3/users => generated 248 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:46:55.616911 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d2494-abde-40a1-92c4-56214ca99cb2 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:55.617203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d2494-abde-40a1-92c4-56214ca99cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:55.617513 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d2494-abde-40a1-92c4-56214ca99cb2 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:55.623270 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 241/475] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:55.625415 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b74bd0-7d01-4878-972f-61d9152f00e9 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:55.625711 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b74bd0-7d01-4878-972f-61d9152f00e9 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:55.626050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b74bd0-7d01-4878-972f-61d9152f00e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:55.626338 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fdcf2-dd35-4fbe-b2bc-aea85786c768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:55.627249 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fdcf2-dd35-4fbe-b2bc-aea85786c768 None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:55.627728 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fdcf2-dd35-4fbe-b2bc-aea85786c768 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:55.628346 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fdcf2-dd35-4fbe-b2bc-aea85786c768 None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:55.629244 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535fdcf2-dd35-4fbe-b2bc-aea85786c768 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:55.632825 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535fdcf2-dd35-4fbe-b2bc-aea85786c768 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:55.651888 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-29b74bd0-7d01-4878-972f-61d9152f00e9 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:55.653183 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-29b74bd0-7d01-4878-972f-61d9152f00e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:55.696896 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:55.723458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 235/476] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 2 08:46:55 2022] POST /identity/v3/users => generated 249 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:46:55.758608 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 242/477] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:55.786708 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fdcf2-dd35-4fbe-b2bc-aea85786c768 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:55.912778 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fdcf2-dd35-4fbe-b2bc-aea85786c768 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:55.953357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa3032-12ad-48a4-84b1-2c2818b831bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:55.954350 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa3032-12ad-48a4-84b1-2c2818b831bd None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:55.954837 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa3032-12ad-48a4-84b1-2c2818b831bd None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:55.955313 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa3032-12ad-48a4-84b1-2c2818b831bd None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:55.956223 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa3032-12ad-48a4-84b1-2c2818b831bd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:55.957811 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afa3032-12ad-48a4-84b1-2c2818b831bd None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:55.993196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.sql.core [None req-6afa3032-12ad-48a4-84b1-2c2818b831bd None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 02 08:46:55.993196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 02 08:46:55.993196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [parameters: {'id': '492a69a03e8642afa426c87428db565a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 02 08:46:55.993196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80329) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 02 08:46:56.012591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-6afa3032-12ad-48a4-84b1-2c2818b831bd None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 02 08:46:56.012591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:56.012591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 08:46:56.012591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 08:46:56.012591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 08:46:56.012591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 08:46:56.012591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 08:46:56.012591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 08:46:56.012591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 08:46:56.012591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application conn.query(q) Aug 02 08:46:56.012591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 08:46:56.012591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 08:46:56.012591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 08:46:56.012591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application result.read() Aug 02 08:46:56.012591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 08:46:56.013741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 08:46:56.013741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 08:46:56.013741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 08:46:56.013741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 08:46:56.013741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 08:46:56.013741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 08:46:56.013741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 08:46:56.013741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 02 08:46:56.013741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:56.013741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 02 08:46:56.013741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:56.013741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:56.013741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 02 08:46:56.013741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 02 08:46:56.013741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 02 08:46:56.013741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return ref.to_dict() Aug 02 08:46:56.013741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 08:46:56.013741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application next(self.gen) Aug 02 08:46:56.014920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 02 08:46:56.014920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application yield resource Aug 02 08:46:56.014920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 08:46:56.014920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application next(self.gen) Aug 02 08:46:56.014920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 02 08:46:56.014920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self.session.rollback() Aug 02 08:46:56.014920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 02 08:46:56.014920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self.force_reraise() Aug 02 08:46:56.014920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 02 08:46:56.014920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise self.value Aug 02 08:46:56.014920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 02 08:46:56.014920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 02 08:46:56.014920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 02 08:46:56.014920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application session.commit() Aug 02 08:46:56.014920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 02 08:46:56.014920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 02 08:46:56.014920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 02 08:46:56.016133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._prepare_impl() Aug 02 08:46:56.016133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 02 08:46:56.016133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self.session.flush() Aug 02 08:46:56.016133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 02 08:46:56.016133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._flush(objects) Aug 02 08:46:56.016133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 02 08:46:56.016133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 02 08:46:56.016133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 02 08:46:56.016133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application compat.raise_( Aug 02 08:46:56.016133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 08:46:56.016133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception Aug 02 08:46:56.016133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 02 08:46:56.016133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application flush_context.execute() Aug 02 08:46:56.016133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 02 08:46:56.016133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rec.execute(self) Aug 02 08:46:56.016133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 02 08:46:56.016133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 02 08:46:56.017531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 02 08:46:56.017531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 02 08:46:56.017531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 02 08:46:56.017531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 02 08:46:56.017531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 02 08:46:56.017531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 02 08:46:56.017531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 02 08:46:56.017531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 02 08:46:56.017531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 02 08:46:56.017531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 02 08:46:56.017531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 02 08:46:56.017531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 02 08:46:56.017531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 02 08:46:56.017531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 02 08:46:56.017531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 08:46:56.017531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception Aug 02 08:46:56.037198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 08:46:56.037198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 08:46:56.037198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 08:46:56.037198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 08:46:56.037198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 08:46:56.037198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 08:46:56.037198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 08:46:56.037198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application conn.query(q) Aug 02 08:46:56.037198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 08:46:56.037198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 08:46:56.037198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 08:46:56.037198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application result.read() Aug 02 08:46:56.037198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 08:46:56.037198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 08:46:56.037198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 08:46:56.037198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 08:46:56.038634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 08:46:56.038634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 08:46:56.038634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 08:46:56.038634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 08:46:56.038634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 02 08:46:56.038634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 02 08:46:56.038634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [parameters: {'id': '492a69a03e8642afa426c87428db565a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 02 08:46:56.038634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 02 08:46:56.038634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:56.038634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:46:56.038634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:56.038634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:56.038634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:56.038634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:56.038634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:56.038634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:56.039446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:56.039446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:56.039446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:56.039446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:56.039446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:56.039446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:56.039446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 02 08:46:56.039446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 02 08:46:56.039446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:56.039446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:56.039446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 02 08:46:56.039446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 02 08:46:56.039446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 02 08:46:56.039446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 02 08:46:56.039446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 02 08:46:56.039446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:56.040292 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 236/478] 10.222.0.46 () {60 vars in 1131 bytes} [Tue Aug 2 08:46:55 2022] POST /identity/v3/roles => generated 144 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:56.040362 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b74bd0-7d01-4878-972f-61d9152f00e9 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:56.144505 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b74bd0-7d01-4878-972f-61d9152f00e9 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:56.187386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373dced-ccf9-4287-b260-5dc27853b212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:56.188378 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373dced-ccf9-4287-b260-5dc27853b212 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:56.188858 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373dced-ccf9-4287-b260-5dc27853b212 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:56.189332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373dced-ccf9-4287-b260-5dc27853b212 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:56.200923 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4373dced-ccf9-4287-b260-5dc27853b212 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:56.206268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-4373dced-ccf9-4287-b260-5dc27853b212 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:56.206268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:56.206268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:56.206268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:56.206268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:56.206268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:56.206268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:56.206268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:56.206268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:56.206268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:56.206268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:56.206268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:56.206268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:56.206268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:56.206268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:56.206268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:56.207154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:56.207154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:56.207154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:56.207154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:56.207154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:56.207154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:56.207154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:56.207154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:56.207154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:56.207154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:56.207154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:56.207154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:56.207154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:56.207154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:56.207154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:56.207154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:56.207957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:56.207957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:56.207957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:56.207957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:56.207957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:56.207957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:56.207957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:56.207957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:56.207957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:56.207957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:56.207957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 08:46:56.207957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:56.213873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 243/479] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 2 08:46:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:56.217767 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa3032-12ad-48a4-84b1-2c2818b831bd None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:56.251537 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:56.328246 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa3032-12ad-48a4-84b1-2c2818b831bd None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:56.365119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef17d80-ff39-4bf5-afd2-2a3ebf51e49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:56.366801 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef17d80-ff39-4bf5-afd2-2a3ebf51e49d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:56.367323 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef17d80-ff39-4bf5-afd2-2a3ebf51e49d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:56.367805 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef17d80-ff39-4bf5-afd2-2a3ebf51e49d None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:56.368773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef17d80-ff39-4bf5-afd2-2a3ebf51e49d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:56.369296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef17d80-ff39-4bf5-afd2-2a3ebf51e49d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:56.370902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef17d80-ff39-4bf5-afd2-2a3ebf51e49d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:56.385918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 237/480] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:46:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:57.161401 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c86ed4-3c2a-4453-8817-289a3636aa97 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:57.161961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c86ed4-3c2a-4453-8817-289a3636aa97 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:57.161961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c86ed4-3c2a-4453-8817-289a3636aa97 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:57.164081 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 244/481] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:57.169330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b025f7-5a0f-4ea1-a348-c9b0a84477e3 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:57.169732 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b025f7-5a0f-4ea1-a348-c9b0a84477e3 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:57.169940 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b025f7-5a0f-4ea1-a348-c9b0a84477e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:57.188391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-e6b025f7-5a0f-4ea1-a348-c9b0a84477e3 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:57.189491 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e6b025f7-5a0f-4ea1-a348-c9b0a84477e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:57.287517 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 238/482] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:57.323474 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c86ed4-3c2a-4453-8817-289a3636aa97 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:57.362245 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:57.453050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c86ed4-3c2a-4453-8817-289a3636aa97 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:57.488224 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ebd228-fd3e-4a8e-975e-6a5cace2398a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:57.488863 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ebd228-fd3e-4a8e-975e-6a5cace2398a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:57.489195 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ebd228-fd3e-4a8e-975e-6a5cace2398a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:57.489476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ebd228-fd3e-4a8e-975e-6a5cace2398a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:57.500473 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ebd228-fd3e-4a8e-975e-6a5cace2398a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:57.505501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-d9ebd228-fd3e-4a8e-975e-6a5cace2398a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:46:57.505501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:57.505501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:57.505501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:57.505501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:57.505501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:57.505501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:57.505501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:57.505501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:57.505501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:57.505501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:57.505501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:57.505501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:57.505501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:57.505501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:57.505501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:57.506773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:57.506773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:57.506773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:57.506773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:57.506773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:57.506773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:57.506773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:57.506773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:57.506773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:57.506773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:57.506773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:57.506773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:57.506773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:57.506773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:57.506773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:57.506773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:57.508135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:57.508135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:57.508135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:57.508135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:57.508135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:57.508135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:57.508135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:57.508135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:57.508135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:57.508135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:57.508135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:46:57.508135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:57.508135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 245/483] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 2 08:46:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:57.512906 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b025f7-5a0f-4ea1-a348-c9b0a84477e3 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:57.534019 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:57.617198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b025f7-5a0f-4ea1-a348-c9b0a84477e3 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:57.659812 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46e201-6289-4e48-ae4f-ff9d91383c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:57.660428 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46e201-6289-4e48-ae4f-ff9d91383c3a None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:57.660710 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46e201-6289-4e48-ae4f-ff9d91383c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:57.660979 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46e201-6289-4e48-ae4f-ff9d91383c3a None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:57.661684 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46e201-6289-4e48-ae4f-ff9d91383c3a None admin] RBAC: Adding query filter params (name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:57.662012 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46e201-6289-4e48-ae4f-ff9d91383c3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:57.663898 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46e201-6289-4e48-ae4f-ff9d91383c3a None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:57.676497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 239/484] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 2 08:46:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:57.682618 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ebd228-fd3e-4a8e-975e-6a5cace2398a None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:57.795574 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:57.809697 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ebd228-fd3e-4a8e-975e-6a5cace2398a None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:57.844695 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6840b-fa8c-495f-9bbb-a00d55327308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:57.845343 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6840b-fa8c-495f-9bbb-a00d55327308 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:57.845620 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6840b-fa8c-495f-9bbb-a00d55327308 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:57.845915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6840b-fa8c-495f-9bbb-a00d55327308 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:57.857733 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff6840b-fa8c-495f-9bbb-a00d55327308 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:57.859241 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff6840b-fa8c-495f-9bbb-a00d55327308 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:57.874476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-5ff6840b-fa8c-495f-9bbb-a00d55327308 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:46:57.874476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:57.874476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:57.874476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:57.874476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:57.874476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:57.874476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:57.874476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:57.874476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:57.874476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:57.874476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:57.874476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:57.874476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:46:57.874476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:46:57.874476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:46:57.875390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:46:57.875390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:57.875390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:57.875390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:57.875390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:57.875390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:57.875390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:57.875390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:57.875390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:57.875390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:57.875390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:57.875390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:57.875390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:57.875390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:57.875390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:57.875390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:57.875390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:57.876838 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:46:57.876838 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:46:57.876838 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:46:57.876838 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:57.876838 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 246/485] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:46:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:57.881331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46e201-6289-4e48-ae4f-ff9d91383c3a None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:58.052849 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:58.074536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46e201-6289-4e48-ae4f-ff9d91383c3a None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:58.113323 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f2650a-a050-4e86-a60d-85c8a7b07365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:58.113955 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2650a-a050-4e86-a60d-85c8a7b07365 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:58.114239 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2650a-a050-4e86-a60d-85c8a7b07365 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:58.114511 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f2650a-a050-4e86-a60d-85c8a7b07365 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:58.115195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f2650a-a050-4e86-a60d-85c8a7b07365 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:58.115497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f2650a-a050-4e86-a60d-85c8a7b07365 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:58.117393 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f2650a-a050-4e86-a60d-85c8a7b07365 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:58.133791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 240/486] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:46:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:58.139182 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6840b-fa8c-495f-9bbb-a00d55327308 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:58.272888 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6840b-fa8c-495f-9bbb-a00d55327308 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:58.294894 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a89e8-e6e1-4b5d-ab71-a37cab082ff4 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:58.295411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a89e8-e6e1-4b5d-ab71-a37cab082ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:58.295673 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a89e8-e6e1-4b5d-ab71-a37cab082ff4 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:58.297989 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 241/487] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:58.302716 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1a94f-ff82-46f1-9999-12b43910938d None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:58.303188 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1a94f-ff82-46f1-9999-12b43910938d None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:58.303545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1a94f-ff82-46f1-9999-12b43910938d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:58.310416 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93824f-c753-4ed2-989c-1a6349ba28d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:58.311332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93824f-c753-4ed2-989c-1a6349ba28d7 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:58.311771 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93824f-c753-4ed2-989c-1a6349ba28d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:58.312188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a93824f-c753-4ed2-989c-1a6349ba28d7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:58.321026 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a93824f-c753-4ed2-989c-1a6349ba28d7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:58.323041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a93824f-c753-4ed2-989c-1a6349ba28d7 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:58.323172 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-e5e1a94f-ff82-46f1-9999-12b43910938d None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:58.324502 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e5e1a94f-ff82-46f1-9999-12b43910938d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:58.326190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:58.340391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-0a93824f-c753-4ed2-989c-1a6349ba28d7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:46:58.340391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:58.340391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:58.340391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:58.340391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:58.340391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:58.340391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:58.340391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:58.340391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:58.340391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:58.340391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:58.340391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:58.340391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:46:58.340391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:46:58.340391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:46:58.340391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:46:58.341924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:58.341924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:58.341924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:46:58.341924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:46:58.341924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:46:58.341924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:46:58.341924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:58.341924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:58.341924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:58.341924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:58.341924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:58.341924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:58.341924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:58.341924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:58.341924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:58.341924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:58.341924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:58.341924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:58.343532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:58.343532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:58.343532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:46:58.343532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:46:58.343532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:46:58.343532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:46:58.343532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:46:58.343532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:46:58.343532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:46:58.343532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:58.345849 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 247/488] 10.222.0.46 () {56 vars in 1093 bytes} [Tue Aug 2 08:46:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:58.351330 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93824f-c753-4ed2-989c-1a6349ba28d7 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:58.409913 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 242/489] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:58.426907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1a94f-ff82-46f1-9999-12b43910938d None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:58.510457 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a93824f-c753-4ed2-989c-1a6349ba28d7 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:58.528635 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:58.539562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1a94f-ff82-46f1-9999-12b43910938d None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:58.546231 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b91927-a599-47ad-bab7-722a720ade48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:58.546852 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b91927-a599-47ad-bab7-722a720ade48 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:58.547133 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b91927-a599-47ad-bab7-722a720ade48 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:58.547410 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b91927-a599-47ad-bab7-722a720ade48 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:58.548202 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b91927-a599-47ad-bab7-722a720ade48 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:58.548509 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b91927-a599-47ad-bab7-722a720ade48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:58.549970 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b91927-a599-47ad-bab7-722a720ade48 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:58.575416 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc644cab-e7c3-4a70-8fb1-ae57ca68a813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:58.576822 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc644cab-e7c3-4a70-8fb1-ae57ca68a813 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:58.577110 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc644cab-e7c3-4a70-8fb1-ae57ca68a813 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:58.577385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc644cab-e7c3-4a70-8fb1-ae57ca68a813 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:58.592001 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 248/490] 10.222.0.46 () {56 vars in 1133 bytes} [Tue Aug 2 08:46:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:58.592001 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d4b5b-d154-42f4-abca-a565052a4dce None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:58.592001 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d4b5b-d154-42f4-abca-a565052a4dce None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:58.592001 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d4b5b-d154-42f4-abca-a565052a4dce None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:58.592001 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 249/491] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:58.592001 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a50bad-ef2c-4eb8-a44e-1434f1049286 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:58.592460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a50bad-ef2c-4eb8-a44e-1434f1049286 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:58.592460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a50bad-ef2c-4eb8-a44e-1434f1049286 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:58.593678 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 250/492] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:58.595557 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc644cab-e7c3-4a70-8fb1-ae57ca68a813 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:58.598078 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a50bad-ef2c-4eb8-a44e-1434f1049286 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:58.601245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-bc644cab-e7c3-4a70-8fb1-ae57ca68a813 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:46:58.601245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:58.601245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:58.601245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:58.601245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:58.601245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:58.601245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:58.601245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:58.601245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:58.601245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:58.601245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:58.601245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:58.601245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:58.601245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:58.601245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:58.601245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:58.602289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:58.602289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:58.602289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:58.602289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:58.602289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:58.602289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:58.602289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:58.602289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:58.602289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:58.602289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:58.602289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:58.602289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:58.602289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:58.602289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:58.602289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:58.602289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:58.603174 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:58.603174 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:58.603174 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:58.603174 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:58.603174 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:58.603174 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:58.603174 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:58.603174 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:58.603174 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:58.603174 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:58.603174 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:46:58.603174 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:58.603174 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 243/493] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 2 08:46:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:58.604226 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7179ad-4c65-42e0-8b08-b3bbf36c9130 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:58.604226 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7179ad-4c65-42e0-8b08-b3bbf36c9130 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:58.604455 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7179ad-4c65-42e0-8b08-b3bbf36c9130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:58.623158 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-9a7179ad-4c65-42e0-8b08-b3bbf36c9130 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:58.624442 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-9a7179ad-4c65-42e0-8b08-b3bbf36c9130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:58.667308 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:58.697438 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 244/494] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:58.706084 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688e498-60d9-4440-ab91-d9f9952cf8b6 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:58.706084 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688e498-60d9-4440-ab91-d9f9952cf8b6 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:58.706084 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688e498-60d9-4440-ab91-d9f9952cf8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:58.718914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-2688e498-60d9-4440-ab91-d9f9952cf8b6 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:58.720192 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-2688e498-60d9-4440-ab91-d9f9952cf8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:58.721489 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a50bad-ef2c-4eb8-a44e-1434f1049286 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:58.762952 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201ff01-9c66-4c41-8e53-284ae392253e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:58.763585 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201ff01-9c66-4c41-8e53-284ae392253e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:58.763871 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201ff01-9c66-4c41-8e53-284ae392253e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:58.764150 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201ff01-9c66-4c41-8e53-284ae392253e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:58.782636 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:58.793979 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 245/495] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:58.794729 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688e498-60d9-4440-ab91-d9f9952cf8b6 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:58.842044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201ff01-9c66-4c41-8e53-284ae392253e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:58.843744 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201ff01-9c66-4c41-8e53-284ae392253e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:58.908002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:58.911950 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688e498-60d9-4440-ab91-d9f9952cf8b6 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:58.954927 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65412fb2-c005-493c-b4e6-0be4d849fbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:58.955553 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65412fb2-c005-493c-b4e6-0be4d849fbb0 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:58.955837 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65412fb2-c005-493c-b4e6-0be4d849fbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:58.960908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65412fb2-c005-493c-b4e6-0be4d849fbb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:58.961063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 251/496] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:46:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:46:58.961159 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65412fb2-c005-493c-b4e6-0be4d849fbb0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:58.961251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201ff01-9c66-4c41-8e53-284ae392253e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:58.961335 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65412fb2-c005-493c-b4e6-0be4d849fbb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:58.961335 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65412fb2-c005-493c-b4e6-0be4d849fbb0 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:58.968890 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 246/497] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 2 08:46:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:58.969685 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65412fb2-c005-493c-b4e6-0be4d849fbb0 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:59.026626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:59.070999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201ff01-9c66-4c41-8e53-284ae392253e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:59.072359 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65412fb2-c005-493c-b4e6-0be4d849fbb0 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:59.107974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01faf4-8931-48c7-a020-dc0bcb54cc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:59.108304 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e5711-d43f-4e9c-89d8-f28fbd6b8851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:59.108545 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01faf4-8931-48c7-a020-dc0bcb54cc3b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:59.108822 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01faf4-8931-48c7-a020-dc0bcb54cc3b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:59.108918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e5711-d43f-4e9c-89d8-f28fbd6b8851 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:59.109098 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01faf4-8931-48c7-a020-dc0bcb54cc3b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:59.109192 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e5711-d43f-4e9c-89d8-f28fbd6b8851 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:59.109405 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e5711-d43f-4e9c-89d8-f28fbd6b8851 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:59.119466 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01faf4-8931-48c7-a020-dc0bcb54cc3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:59.119865 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e5711-d43f-4e9c-89d8-f28fbd6b8851 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:46:59.124381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-4c01faf4-8931-48c7-a020-dc0bcb54cc3b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:46:59.124381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:59.124381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:59.124381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:59.124381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:59.124381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:59.124381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:59.124381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:59.124381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:59.124381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:59.124381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:59.124381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:59.124381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:59.124381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:59.124381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:59.124381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:59.125238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:59.125238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:59.125238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:59.125238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:59.125238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:59.125238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:59.125238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:59.125238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:59.125238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:59.125238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:59.125238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:59.125238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:59.125238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:59.125238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:59.125238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:59.125238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:59.126064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:59.126064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:59.126064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:59.126064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:59.126064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:59.126064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:59.126064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:59.126064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:59.126064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:59.126064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:59.126064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:46:59.126064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:59.126675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-1b7e5711-d43f-4e9c-89d8-f28fbd6b8851 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:46:59.126675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:59.126675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:59.126675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:59.126675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:59.126675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:59.126675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:59.126675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:59.126675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:59.126675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:59.126675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:59.126675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:59.126675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:46:59.126675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:46:59.126675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:46:59.126675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:46:59.127464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:46:59.127464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:46:59.127464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:46:59.127464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:46:59.127464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:59.127464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:59.127464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:59.127464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:59.127464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:59.127464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:59.127464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:59.127464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:59.127464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:59.127464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:59.127464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:59.127464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:59.128252 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:59.128252 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:59.128252 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:59.128252 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:59.128252 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:46:59.128252 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:46:59.128252 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:46:59.128252 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:46:59.128252 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:46:59.128252 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:46:59.128252 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:46:59.128252 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:59.128853 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 252/498] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 2 08:46:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:59.128853 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01faf4-8931-48c7-a020-dc0bcb54cc3b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:59.136397 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 247/499] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 2 08:46:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:59.136397 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e5711-d43f-4e9c-89d8-f28fbd6b8851 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:59.153295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:59.232145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e5711-d43f-4e9c-89d8-f28fbd6b8851 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:59.232326 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01faf4-8931-48c7-a020-dc0bcb54cc3b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:59.246380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:59.265654 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8d6a3-bfe7-481e-a109-e33518301a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:59.265833 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ecaaa7-cc78-40a3-a6ab-8b2d81292eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:59.266321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8d6a3-bfe7-481e-a109-e33518301a97 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:59.266440 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ecaaa7-cc78-40a3-a6ab-8b2d81292eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:59.266601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8d6a3-bfe7-481e-a109-e33518301a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:59.266692 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ecaaa7-cc78-40a3-a6ab-8b2d81292eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:59.266873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8d6a3-bfe7-481e-a109-e33518301a97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:59.266961 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ecaaa7-cc78-40a3-a6ab-8b2d81292eb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:59.277463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8d6a3-bfe7-481e-a109-e33518301a97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:59.278936 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ecaaa7-cc78-40a3-a6ab-8b2d81292eb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:59.279036 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8d6a3-bfe7-481e-a109-e33518301a97 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:59.280383 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ecaaa7-cc78-40a3-a6ab-8b2d81292eb0 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:59.293155 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-54e8d6a3-bfe7-481e-a109-e33518301a97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:46:59.293155 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:59.293155 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:59.293155 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:59.293155 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:59.293155 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:59.293155 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:59.293155 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:59.293155 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:59.293155 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:59.293155 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:59.293155 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:59.293155 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:46:59.293155 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:46:59.293155 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:46:59.294052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:46:59.294052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:59.294052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:59.294052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:59.294052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:59.294052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:59.294052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:59.294052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:59.294052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:59.294052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:59.294052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:59.294052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:59.294052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:59.294052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:59.294052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:59.294052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:59.294052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:59.294893 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:46:59.294893 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:46:59.294893 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:46:59.294893 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:59.295111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-09ecaaa7-cc78-40a3-a6ab-8b2d81292eb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:46:59.295111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:59.295111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:59.295111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:59.295111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:59.295111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:59.295111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:59.295111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:59.295111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:59.295111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:59.295111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:59.295111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:59.295111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:46:59.295111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:46:59.295111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:46:59.296157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:46:59.296157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:59.296157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:59.296157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:59.296157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:59.296157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:59.296157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:59.296157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:59.296157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:59.296157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:59.296157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:59.296157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:59.296157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:59.296157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:59.296157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:59.296157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:59.296157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:59.296992 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:46:59.296992 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:46:59.296992 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:46:59.296992 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:59.296992 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 248/500] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:59.296992 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ecaaa7-cc78-40a3-a6ab-8b2d81292eb0 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:59.298388 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 253/501] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:59.299828 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34977e23-4f8a-4e96-9411-ad349f27fba6 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:59.300109 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34977e23-4f8a-4e96-9411-ad349f27fba6 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:59.300387 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34977e23-4f8a-4e96-9411-ad349f27fba6 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:59.301674 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 254/502] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:46:59.302388 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34977e23-4f8a-4e96-9411-ad349f27fba6 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:59.374009 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:59.398806 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ecaaa7-cc78-40a3-a6ab-8b2d81292eb0 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:59.404987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34977e23-4f8a-4e96-9411-ad349f27fba6 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:59.432952 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac362b-5bb9-4cd2-b8c7-a837d544ff21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:59.433571 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac362b-5bb9-4cd2-b8c7-a837d544ff21 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:59.433836 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac362b-5bb9-4cd2-b8c7-a837d544ff21 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:59.434138 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ac362b-5bb9-4cd2-b8c7-a837d544ff21 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:59.434839 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac362b-5bb9-4cd2-b8c7-a837d544ff21 None admin] RBAC: Adding query filter params (name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:59.435142 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac362b-5bb9-4cd2-b8c7-a837d544ff21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:59.436497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ac362b-5bb9-4cd2-b8c7-a837d544ff21 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:59.438780 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f72549-60de-433c-9a2d-04db7785aa5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:59.439388 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f72549-60de-433c-9a2d-04db7785aa5d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:59.439667 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f72549-60de-433c-9a2d-04db7785aa5d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:59.439942 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f72549-60de-433c-9a2d-04db7785aa5d None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:59.440645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f72549-60de-433c-9a2d-04db7785aa5d None admin] RBAC: Adding query filter params (name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:59.440948 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f72549-60de-433c-9a2d-04db7785aa5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:59.442342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f72549-60de-433c-9a2d-04db7785aa5d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:59.449350 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 249/503] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 2 08:46:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:59.449350 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac362b-5bb9-4cd2-b8c7-a837d544ff21 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:59.455903 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 255/504] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 2 08:46:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:59.456699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f72549-60de-433c-9a2d-04db7785aa5d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:59.477457 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:59.547686 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ac362b-5bb9-4cd2-b8c7-a837d544ff21 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:59.554325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f72549-60de-433c-9a2d-04db7785aa5d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:59.581356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f78899-8c17-4a4b-8412-61e3ac199de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:59.582306 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f78899-8c17-4a4b-8412-61e3ac199de9 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:59.582590 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f78899-8c17-4a4b-8412-61e3ac199de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:59.582863 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f78899-8c17-4a4b-8412-61e3ac199de9 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:59.583541 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f78899-8c17-4a4b-8412-61e3ac199de9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:59.583846 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f78899-8c17-4a4b-8412-61e3ac199de9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:59.585193 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:59.585719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f78899-8c17-4a4b-8412-61e3ac199de9 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:59.589793 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357e736-9d05-4088-be38-42a260f4176e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:59.590431 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357e736-9d05-4088-be38-42a260f4176e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:59.590711 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357e736-9d05-4088-be38-42a260f4176e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:59.590986 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357e736-9d05-4088-be38-42a260f4176e None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:59.591671 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357e736-9d05-4088-be38-42a260f4176e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:46:59.591982 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357e736-9d05-4088-be38-42a260f4176e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:59.593445 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357e736-9d05-4088-be38-42a260f4176e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:59.596714 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 250/505] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:59.600941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f46e1f-57d1-48e1-b760-a0aaa54fe657 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:59.600941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f46e1f-57d1-48e1-b760-a0aaa54fe657 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:59.600941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f46e1f-57d1-48e1-b760-a0aaa54fe657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:59.606274 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 256/506] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:46:59.607115 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357e736-9d05-4088-be38-42a260f4176e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:59.616183 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-27f46e1f-57d1-48e1-b760-a0aaa54fe657 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:46:59.617383 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-27f46e1f-57d1-48e1-b760-a0aaa54fe657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:46:59.683320 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 251/507] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:46:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:46:59.684346 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f46e1f-57d1-48e1-b760-a0aaa54fe657 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:59.708953 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357e736-9d05-4088-be38-42a260f4176e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:59.710353 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:59.742877 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21698483-2e24-4db7-8436-089ccc4271ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:59.743578 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21698483-2e24-4db7-8436-089ccc4271ee None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:59.743854 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21698483-2e24-4db7-8436-089ccc4271ee None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:59.744133 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21698483-2e24-4db7-8436-089ccc4271ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:59.753921 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21698483-2e24-4db7-8436-089ccc4271ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:59.755368 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21698483-2e24-4db7-8436-089ccc4271ee None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:59.770697 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-21698483-2e24-4db7-8436-089ccc4271ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:46:59.770697 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:59.770697 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:59.770697 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:59.770697 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:59.770697 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:59.770697 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:59.770697 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:59.770697 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:59.770697 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:59.770697 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:59.770697 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:59.770697 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:46:59.770697 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:46:59.770697 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:46:59.771703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:46:59.771703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:59.771703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:59.771703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:59.771703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:59.771703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:59.771703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:59.771703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:59.771703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:59.771703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:59.771703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:59.771703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:59.771703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:59.771703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:59.771703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:59.771703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:59.771703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:59.773218 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:46:59.773218 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:46:59.773218 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:46:59.773218 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:59.773218 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 257/508] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:59.773218 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21698483-2e24-4db7-8436-089ccc4271ee None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:59.794430 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f46e1f-57d1-48e1-b760-a0aaa54fe657 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:59.818706 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:59.828445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77faa23-ec76-4149-99e5-c50321fa3c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:59.834393 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77faa23-ec76-4149-99e5-c50321fa3c6b None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:59.834393 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77faa23-ec76-4149-99e5-c50321fa3c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:59.834393 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77faa23-ec76-4149-99e5-c50321fa3c6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:59.840243 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77faa23-ec76-4149-99e5-c50321fa3c6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:59.841740 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77faa23-ec76-4149-99e5-c50321fa3c6b None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:59.857624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-f77faa23-ec76-4149-99e5-c50321fa3c6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:46:59.857624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:59.857624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:59.857624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:59.857624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:59.857624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:59.857624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:59.857624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:59.857624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:59.857624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:59.857624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:59.857624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:59.857624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:46:59.857624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:46:59.857624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:46:59.858573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:46:59.858573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:59.858573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:59.858573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:59.858573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:59.858573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:59.858573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:59.858573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:59.858573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:59.858573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:59.858573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:59.858573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:59.858573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:59.858573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:59.858573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:59.858573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:59.858573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:59.859421 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:46:59.859421 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:46:59.859421 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:46:59.859421 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:46:59.859421 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 252/509] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:59.859691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77faa23-ec76-4149-99e5-c50321fa3c6b None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:59.877682 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21698483-2e24-4db7-8436-089ccc4271ee None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:59.910990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69742c0d-cae7-4f43-b9b5-13ba40ce9b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:59.911626 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69742c0d-cae7-4f43-b9b5-13ba40ce9b48 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:59.911911 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69742c0d-cae7-4f43-b9b5-13ba40ce9b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:59.912190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69742c0d-cae7-4f43-b9b5-13ba40ce9b48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:46:59.918757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69742c0d-cae7-4f43-b9b5-13ba40ce9b48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:46:59.920202 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69742c0d-cae7-4f43-b9b5-13ba40ce9b48 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:46:59.929912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:46:59.934512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-69742c0d-cae7-4f43-b9b5-13ba40ce9b48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:46:59.934512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:46:59.934512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:46:59.934512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:46:59.934512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:46:59.934512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:46:59.934512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:46:59.934512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:46:59.934512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:46:59.934512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:46:59.934512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:46:59.934512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:46:59.934512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:46:59.934512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:46:59.934512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:46:59.935575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:46:59.935575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:46:59.935575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:46:59.935575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:46:59.935575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:46:59.935575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:46:59.935575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:46:59.935575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:46:59.935575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:46:59.935575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:46:59.935575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:46:59.935575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:46:59.935575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:46:59.935575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:46:59.935575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:46:59.935575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:46:59.935575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:46:59.947030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:46:59.947030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:46:59.947030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:46:59.947030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:46:59.947030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:46:59.947030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:46:59.947030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:46:59.947030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:46:59.947030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 258/510] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:46:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:46:59.947030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69742c0d-cae7-4f43-b9b5-13ba40ce9b48 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:46:59.962715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77faa23-ec76-4149-99e5-c50321fa3c6b None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:46:59.995745 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd9055-2fea-47e4-9dff-5e6e92ef8c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:46:59.996463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd9055-2fea-47e4-9dff-5e6e92ef8c81 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:46:59.996819 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd9055-2fea-47e4-9dff-5e6e92ef8c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:46:59.997091 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd9055-2fea-47e4-9dff-5e6e92ef8c81 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:00.004493 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fd9055-2fea-47e4-9dff-5e6e92ef8c81 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:00.006073 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fd9055-2fea-47e4-9dff-5e6e92ef8c81 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:00.018848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-57fd9055-2fea-47e4-9dff-5e6e92ef8c81 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 02 08:47:00.018848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:00.018848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:00.018848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:00.018848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:00.018848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:00.018848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:00.018848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:00.018848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:00.018848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:00.018848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:00.018848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:00.018848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:00.018848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:00.018848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:00.018848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:00.019942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:00.019942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:00.019942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:00.019942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:00.019942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:00.019942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:00.019942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:00.019942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:00.019942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:00.019942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:00.019942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:00.019942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:00.019942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:00.019942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:00.019942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:00.019942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:00.019942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:00.019942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:00.020990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:00.020990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:00.020990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:00.020990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:00.020990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:00.020990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:00.020990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:00.020990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:00.020990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 02 08:47:00.020990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:00.020990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 253/511] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:46:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:00.020990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd9055-2fea-47e4-9dff-5e6e92ef8c81 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:00.042488 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69742c0d-cae7-4f43-b9b5-13ba40ce9b48 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:00.044778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:00.077868 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b26303-e04c-4685-89e1-2812827b4f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:00.078538 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b26303-e04c-4685-89e1-2812827b4f72 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:00.078828 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b26303-e04c-4685-89e1-2812827b4f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:00.079112 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b26303-e04c-4685-89e1-2812827b4f72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:00.090018 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b26303-e04c-4685-89e1-2812827b4f72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:00.091486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b26303-e04c-4685-89e1-2812827b4f72 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:00.105384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-75b26303-e04c-4685-89e1-2812827b4f72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:00.105384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:00.105384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:00.105384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:00.105384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:00.105384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:00.105384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:00.105384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:00.105384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:00.105384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:00.105384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:00.105384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:00.105384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:00.105384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:00.105384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:00.106321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:00.106321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:00.106321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:00.106321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:00.106321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:00.106321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:00.106321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:00.106321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:00.106321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:00.106321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:00.106321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:00.106321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:00.106321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:00.106321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:00.106321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:00.106321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:00.106321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:00.107204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:00.107204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:00.107204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:00.107204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:00.107204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 259/512] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:46:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:00.107466 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b26303-e04c-4685-89e1-2812827b4f72 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:00.122720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd9055-2fea-47e4-9dff-5e6e92ef8c81 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:00.150984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:00.157086 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2d1b9-f765-4d06-9acf-90f724037e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:00.157698 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2d1b9-f765-4d06-9acf-90f724037e39 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:00.158062 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2d1b9-f765-4d06-9acf-90f724037e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:00.158298 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2d1b9-f765-4d06-9acf-90f724037e39 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:00.163478 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2d1b9-f765-4d06-9acf-90f724037e39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:00.163478 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2d1b9-f765-4d06-9acf-90f724037e39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:00.163478 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2d1b9-f765-4d06-9acf-90f724037e39 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:00.172324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 254/513] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:00.173112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2d1b9-f765-4d06-9acf-90f724037e39 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:00.211816 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b26303-e04c-4685-89e1-2812827b4f72 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:00.244572 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14489a81-70ba-47a1-92c1-09fd8184f3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:00.245187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14489a81-70ba-47a1-92c1-09fd8184f3b3 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:00.245475 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14489a81-70ba-47a1-92c1-09fd8184f3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:00.245754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14489a81-70ba-47a1-92c1-09fd8184f3b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:00.246475 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14489a81-70ba-47a1-92c1-09fd8184f3b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:00.246785 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14489a81-70ba-47a1-92c1-09fd8184f3b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:00.248162 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14489a81-70ba-47a1-92c1-09fd8184f3b3 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:00.260092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 260/514] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:00.260881 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14489a81-70ba-47a1-92c1-09fd8184f3b3 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:00.270101 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2d1b9-f765-4d06-9acf-90f724037e39 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:00.281910 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:00.304604 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61bc01-bd60-4cc1-8990-e2d5c65ca798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:00.305278 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61bc01-bd60-4cc1-8990-e2d5c65ca798 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:00.305481 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61bc01-bd60-4cc1-8990-e2d5c65ca798 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:00.305753 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61bc01-bd60-4cc1-8990-e2d5c65ca798 None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:00.306527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb61bc01-bd60-4cc1-8990-e2d5c65ca798 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:00.306838 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb61bc01-bd60-4cc1-8990-e2d5c65ca798 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:00.308274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb61bc01-bd60-4cc1-8990-e2d5c65ca798 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:00.326617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 255/515] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:00.327415 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61bc01-bd60-4cc1-8990-e2d5c65ca798 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:00.366854 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14489a81-70ba-47a1-92c1-09fd8184f3b3 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:00.380354 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:00.402766 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e8010-7f56-4c8f-bc1f-6efe177cd1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:00.403388 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e8010-7f56-4c8f-bc1f-6efe177cd1bd None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:00.403672 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e8010-7f56-4c8f-bc1f-6efe177cd1bd None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:00.403959 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9e8010-7f56-4c8f-bc1f-6efe177cd1bd None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:00.404747 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e8010-7f56-4c8f-bc1f-6efe177cd1bd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:00.405059 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e8010-7f56-4c8f-bc1f-6efe177cd1bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:00.406640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9e8010-7f56-4c8f-bc1f-6efe177cd1bd None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:00.424253 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 261/516] 10.222.0.46 () {56 vars in 1137 bytes} [Tue Aug 2 08:47:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:00.425067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e8010-7f56-4c8f-bc1f-6efe177cd1bd None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:00.432742 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61bc01-bd60-4cc1-8990-e2d5c65ca798 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:00.466220 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c8476-7a25-4013-bd83-29841d09c2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:00.466769 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c8476-7a25-4013-bd83-29841d09c2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:00.467053 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c8476-7a25-4013-bd83-29841d09c2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:00.467329 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c8476-7a25-4013-bd83-29841d09c2f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:00.468018 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c8476-7a25-4013-bd83-29841d09c2f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:00.468324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c8476-7a25-4013-bd83-29841d09c2f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:00.469706 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c8476-7a25-4013-bd83-29841d09c2f0 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:00.481847 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 256/517] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:00.482654 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c8476-7a25-4013-bd83-29841d09c2f0 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:00.500179 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:00.523756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9e8010-7f56-4c8f-bc1f-6efe177cd1bd None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:00.558197 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f924729-562c-419e-bd01-1fe0e77f91f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:00.558841 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f924729-562c-419e-bd01-1fe0e77f91f4 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:00.559123 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f924729-562c-419e-bd01-1fe0e77f91f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:00.559400 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f924729-562c-419e-bd01-1fe0e77f91f4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:00.566574 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f924729-562c-419e-bd01-1fe0e77f91f4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:00.568015 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f924729-562c-419e-bd01-1fe0e77f91f4 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:00.579593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-1f924729-562c-419e-bd01-1fe0e77f91f4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 08:47:00.579593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:00.579593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:00.579593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:00.579593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:00.579593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:00.579593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:00.579593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:00.579593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:00.579593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:00.579593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:00.579593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:00.579593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:00.579593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:00.579593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:00.579593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:00.580559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:00.580559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:00.580559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:00.580559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:00.580559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:00.580559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:00.580559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:00.580559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:00.580559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:00.580559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:00.580559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:00.580559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:00.580559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:00.580559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:00.580559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:00.580559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:00.580559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:00.580559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:00.581468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:00.581468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:00.581468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:00.581468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:00.581468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:00.581468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:00.581468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:00.581468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:00.581468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 08:47:00.581468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:00.581468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 262/518] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:47:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:00.581468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f924729-562c-419e-bd01-1fe0e77f91f4 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:00.582644 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c8476-7a25-4013-bd83-29841d09c2f0 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:00.594354 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:00.618119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d9e3e-169d-43d5-931c-3825c7d686e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:00.618800 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d9e3e-169d-43d5-931c-3825c7d686e3 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:00.619078 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d9e3e-169d-43d5-931c-3825c7d686e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:00.619346 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d9e3e-169d-43d5-931c-3825c7d686e3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:00.626754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d9e3e-169d-43d5-931c-3825c7d686e3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:00.628193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d9e3e-169d-43d5-931c-3825c7d686e3 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:00.639489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-9d0d9e3e-169d-43d5-931c-3825c7d686e3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 08:47:00.639489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:00.639489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:00.639489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:00.639489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:00.639489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:00.639489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:00.639489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:00.639489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:00.639489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:00.639489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:00.639489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:00.639489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:00.639489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:00.639489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:00.639489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:00.640417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:00.640417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:00.640417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:00.640417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:00.640417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:00.640417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:00.640417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:00.640417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:00.640417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:00.640417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:00.640417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:00.640417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:00.640417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:00.640417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:00.640417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:00.640417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:00.640417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:00.640417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:00.641351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:00.641351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:00.641351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:00.641351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:00.641351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:00.641351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:00.641351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:00.641351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:00.641351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 08:47:00.641351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:00.641351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 257/519] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 2 08:47:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:00.641351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d9e3e-169d-43d5-931c-3825c7d686e3 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:00.684870 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f924729-562c-419e-bd01-1fe0e77f91f4 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:00.716974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee8220-774d-482c-8565-1316bc0279d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:00.717500 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee8220-774d-482c-8565-1316bc0279d6 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:00.717777 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee8220-774d-482c-8565-1316bc0279d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:00.718097 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee8220-774d-482c-8565-1316bc0279d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:00.718879 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee8220-774d-482c-8565-1316bc0279d6 None admin] RBAC: Adding query filter params (role.id=391becda2bf04438b8d1763ab99acf1b, scope.project.id=4227a5b804c0414c99d407bb1415cf49, user.id=3191d804e8dc4b1a85f490319833ec07) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:00.719184 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee8220-774d-482c-8565-1316bc0279d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:00.720597 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee8220-774d-482c-8565-1316bc0279d6 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:00.722325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:00.726755 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 263/520] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:47:00 2022] GET /identity/v3/role_assignments?user.id=3191d804e8dc4b1a85f490319833ec07&scope.project.id=4227a5b804c0414c99d407bb1415cf49&role.id=391becda2bf04438b8d1763ab99acf1b => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:00.727511 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee8220-774d-482c-8565-1316bc0279d6 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:00.739709 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d9e3e-169d-43d5-931c-3825c7d686e3 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:00.786277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148c1a6-93b1-42ae-9fd0-498a86017805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:00.786919 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148c1a6-93b1-42ae-9fd0-498a86017805 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:00.787206 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148c1a6-93b1-42ae-9fd0-498a86017805 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:00.787484 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148c1a6-93b1-42ae-9fd0-498a86017805 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:00.834892 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:00.846761 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee8220-774d-482c-8565-1316bc0279d6 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:00.868442 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148c1a6-93b1-42ae-9fd0-498a86017805 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:00.870092 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a148c1a6-93b1-42ae-9fd0-498a86017805 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:00.886517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cde04-1ef7-469c-b5ef-6deebd1c600f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:00.887453 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cde04-1ef7-469c-b5ef-6deebd1c600f None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:00.887745 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cde04-1ef7-469c-b5ef-6deebd1c600f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:00.888024 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cde04-1ef7-469c-b5ef-6deebd1c600f None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:00.888715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966cde04-1ef7-469c-b5ef-6deebd1c600f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:00.891100 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966cde04-1ef7-469c-b5ef-6deebd1c600f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:00.978829 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 258/521] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:47:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:47:00.979681 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148c1a6-93b1-42ae-9fd0-498a86017805 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:00.999213 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 264/522] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 2 08:47:00 2022] POST /identity/v3/users => generated 247 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:47:01.000329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cde04-1ef7-469c-b5ef-6deebd1c600f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:01.072600 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:01.083123 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:01.105402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148c1a6-93b1-42ae-9fd0-498a86017805 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:01.124032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cde04-1ef7-469c-b5ef-6deebd1c600f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:01.145307 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9abbe-1e06-4109-b37c-c9d6b56dce02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:01.146505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9abbe-1e06-4109-b37c-c9d6b56dce02 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:01.147741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9abbe-1e06-4109-b37c-c9d6b56dce02 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:01.147741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b9abbe-1e06-4109-b37c-c9d6b56dce02 None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:01.148679 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9abbe-1e06-4109-b37c-c9d6b56dce02 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:01.149086 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9abbe-1e06-4109-b37c-c9d6b56dce02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:01.150737 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b9abbe-1e06-4109-b37c-c9d6b56dce02 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:01.164153 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad15fa-8c10-4983-b4f7-aaeee9c77110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:01.165085 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad15fa-8c10-4983-b4f7-aaeee9c77110 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:01.165524 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad15fa-8c10-4983-b4f7-aaeee9c77110 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:01.165965 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad15fa-8c10-4983-b4f7-aaeee9c77110 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:01.166982 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad15fa-8c10-4983-b4f7-aaeee9c77110 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:01.167462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad15fa-8c10-4983-b4f7-aaeee9c77110 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:01.169075 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad15fa-8c10-4983-b4f7-aaeee9c77110 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:01.190741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 265/523] 10.222.0.46 () {56 vars in 1139 bytes} [Tue Aug 2 08:47:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:01.191534 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad15fa-8c10-4983-b4f7-aaeee9c77110 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:01.202291 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 259/524] 10.222.0.46 () {56 vars in 1137 bytes} [Tue Aug 2 08:47:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:01.202291 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9abbe-1e06-4109-b37c-c9d6b56dce02 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:01.315820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad15fa-8c10-4983-b4f7-aaeee9c77110 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:01.331034 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b9abbe-1e06-4109-b37c-c9d6b56dce02 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:01.333503 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:01.360284 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d6b15-c8a7-433f-acd6-12a9c709b8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:01.361183 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d6b15-c8a7-433f-acd6-12a9c709b8bd None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:01.361716 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d6b15-c8a7-433f-acd6-12a9c709b8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:01.362119 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d6b15-c8a7-433f-acd6-12a9c709b8bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:01.366737 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:01.376591 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d6b15-c8a7-433f-acd6-12a9c709b8bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:01.381981 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d6b15-c8a7-433f-acd6-12a9c709b8bd None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:01.382258 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fee1e-eb3f-474c-9a9d-1b60d25f7928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:01.382919 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fee1e-eb3f-474c-9a9d-1b60d25f7928 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:01.383266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fee1e-eb3f-474c-9a9d-1b60d25f7928 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:01.383724 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63fee1e-eb3f-474c-9a9d-1b60d25f7928 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:01.399104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-318d6b15-c8a7-433f-acd6-12a9c709b8bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:01.399104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:01.399104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:01.399104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:01.399104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:01.399104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:01.399104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:01.399104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:01.399104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:01.399104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:01.399104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:01.399104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:01.399104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:01.399104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:01.399104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:01.400000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:01.400000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:01.400000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:01.400000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:01.400000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:01.400000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:01.400000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:01.400000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:01.400000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:01.400000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:01.400000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:01.400000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:01.400000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:01.400000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:01.400000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:01.400000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:01.400000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:01.400856 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:01.400856 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:01.400856 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:01.400856 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:01.403406 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 266/525] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:01.404387 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d6b15-c8a7-433f-acd6-12a9c709b8bd None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:01.466750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fee1e-eb3f-474c-9a9d-1b60d25f7928 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:01.468475 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63fee1e-eb3f-474c-9a9d-1b60d25f7928 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:01.536997 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d6b15-c8a7-433f-acd6-12a9c709b8bd None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:01.575257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919ddc7-af21-4172-990c-c9d337aba563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:01.576201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919ddc7-af21-4172-990c-c9d337aba563 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:01.576673 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919ddc7-af21-4172-990c-c9d337aba563 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:01.577147 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9919ddc7-af21-4172-990c-c9d337aba563 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:01.597922 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 260/526] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:47:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:47:01.597922 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fee1e-eb3f-474c-9a9d-1b60d25f7928 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:01.615596 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:01.642305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:01.657175 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919ddc7-af21-4172-990c-c9d337aba563 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:01.659020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9919ddc7-af21-4172-990c-c9d337aba563 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:01.713182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63fee1e-eb3f-474c-9a9d-1b60d25f7928 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:01.749019 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06609f07-2dab-4781-bfe5-6d737f902036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:01.750001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06609f07-2dab-4781-bfe5-6d737f902036 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:01.750616 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06609f07-2dab-4781-bfe5-6d737f902036 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:01.751181 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06609f07-2dab-4781-bfe5-6d737f902036 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:01.752054 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06609f07-2dab-4781-bfe5-6d737f902036 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:01.752551 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06609f07-2dab-4781-bfe5-6d737f902036 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:01.754318 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06609f07-2dab-4781-bfe5-6d737f902036 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:01.771651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 267/527] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:47:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:47:01.771651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919ddc7-af21-4172-990c-c9d337aba563 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:01.772962 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 261/528] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:01.774073 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06609f07-2dab-4781-bfe5-6d737f902036 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:01.846404 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:01.871224 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:01.883926 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06609f07-2dab-4781-bfe5-6d737f902036 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:01.884643 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9919ddc7-af21-4172-990c-c9d337aba563 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:01.919723 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8728da3-d198-4976-9071-c6a4a831c5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:01.920370 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8728da3-d198-4976-9071-c6a4a831c5af None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:01.920651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8728da3-d198-4976-9071-c6a4a831c5af None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:01.920922 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8728da3-d198-4976-9071-c6a4a831c5af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:01.923064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87149a44-ec0d-41dd-a288-84be115af188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:01.923970 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87149a44-ec0d-41dd-a288-84be115af188 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:01.924457 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87149a44-ec0d-41dd-a288-84be115af188 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:01.927056 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87149a44-ec0d-41dd-a288-84be115af188 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:01.938983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8728da3-d198-4976-9071-c6a4a831c5af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:01.940892 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8728da3-d198-4976-9071-c6a4a831c5af None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:01.944110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87149a44-ec0d-41dd-a288-84be115af188 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:01.946097 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87149a44-ec0d-41dd-a288-84be115af188 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:01.959177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-c8728da3-d198-4976-9071-c6a4a831c5af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:01.959177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:01.959177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:01.959177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:01.959177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:01.959177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:01.959177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:01.959177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:01.959177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:01.959177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:01.959177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:01.959177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:01.959177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:01.959177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:01.959177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:01.960773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:01.960773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:01.960773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:01.960773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:01.960773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:01.960773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:01.960773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:01.960773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:01.960773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:01.960773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:01.960773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:01.960773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:01.960773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:01.960773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:01.960773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:01.960773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:01.960773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:01.961878 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:01.961878 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:01.961878 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:01.961878 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:01.962596 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 262/529] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:01.963439 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8728da3-d198-4976-9071-c6a4a831c5af None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:01.965776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-87149a44-ec0d-41dd-a288-84be115af188 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:01.965776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:01.965776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:01.965776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:01.965776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:01.965776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:01.965776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:01.965776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:01.965776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:01.965776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:01.965776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:01.965776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:01.965776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:01.965776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:01.965776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:01.966936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:01.966936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:01.966936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:01.966936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:01.966936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:01.966936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:01.966936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:01.966936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:01.966936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:01.966936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:01.966936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:01.966936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:01.966936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:01.966936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:01.966936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:01.966936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:01.966936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:01.967799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:01.967799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:01.967799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:01.967799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:01.970015 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 268/530] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:01.971026 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87149a44-ec0d-41dd-a288-84be115af188 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:02.086945 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8728da3-d198-4976-9071-c6a4a831c5af None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:02.104796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87149a44-ec0d-41dd-a288-84be115af188 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:02.121433 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d97f6-a06b-4db3-aa1d-932bd7e81def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:02.122095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d97f6-a06b-4db3-aa1d-932bd7e81def None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:02.122382 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d97f6-a06b-4db3-aa1d-932bd7e81def None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:02.122878 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d97f6-a06b-4db3-aa1d-932bd7e81def None admin] PATH_INFO: `/v3/projects/service` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:02.129469 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d97f6-a06b-4db3-aa1d-932bd7e81def None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:02.130255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:02.130967 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d97f6-a06b-4db3-aa1d-932bd7e81def None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:02.134089 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:02.144569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-478d97f6-a06b-4db3-aa1d-932bd7e81def None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:02.144569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:02.144569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:02.144569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:02.144569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:02.144569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:02.144569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:02.144569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:02.144569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:02.144569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:02.144569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:02.144569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:02.144569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:02.144569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:02.144569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:02.152169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:02.152169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:02.152169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:02.152169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:02.152169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:02.152169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:02.152169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:02.152169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:02.152169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:02.152169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:02.152169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:02.152169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:02.152169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:02.152169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:02.152169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:02.152169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:02.152169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:02.153236 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:02.153236 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:02.153236 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:02.153236 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:02.153236 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:02.153236 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:02.153236 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:02.153236 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:02.153236 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 263/531] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:02.153709 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974746d-f557-475c-a50a-f2ce0f5544e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:02.153709 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974746d-f557-475c-a50a-f2ce0f5544e2 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:02.153709 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974746d-f557-475c-a50a-f2ce0f5544e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:02.153709 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974746d-f557-475c-a50a-f2ce0f5544e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:02.154009 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974746d-f557-475c-a50a-f2ce0f5544e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:02.154009 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974746d-f557-475c-a50a-f2ce0f5544e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:02.154009 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974746d-f557-475c-a50a-f2ce0f5544e2 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:02.154180 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d97f6-a06b-4db3-aa1d-932bd7e81def None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:02.166941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 269/532] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:02.167795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974746d-f557-475c-a50a-f2ce0f5544e2 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:02.267318 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d97f6-a06b-4db3-aa1d-932bd7e81def None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:02.271658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974746d-f557-475c-a50a-f2ce0f5544e2 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:02.304499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70862269-cd89-4e66-aee3-95d1764268cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:02.305145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70862269-cd89-4e66-aee3-95d1764268cd None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:02.305423 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70862269-cd89-4e66-aee3-95d1764268cd None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:02.305700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70862269-cd89-4e66-aee3-95d1764268cd None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:02.306421 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70862269-cd89-4e66-aee3-95d1764268cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:02.306723 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70862269-cd89-4e66-aee3-95d1764268cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:02.308387 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70862269-cd89-4e66-aee3-95d1764268cd None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:02.310409 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf71bf-d56b-4ba5-af3d-df6311c4f419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:02.311474 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf71bf-d56b-4ba5-af3d-df6311c4f419 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:02.311757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf71bf-d56b-4ba5-af3d-df6311c4f419 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:02.312035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf71bf-d56b-4ba5-af3d-df6311c4f419 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:02.312861 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf71bf-d56b-4ba5-af3d-df6311c4f419 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:02.313206 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf71bf-d56b-4ba5-af3d-df6311c4f419 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:02.314674 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf71bf-d56b-4ba5-af3d-df6311c4f419 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:02.325977 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 264/533] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:02.326885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70862269-cd89-4e66-aee3-95d1764268cd None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:02.337270 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 270/534] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:02.338150 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf71bf-d56b-4ba5-af3d-df6311c4f419 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:02.394367 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:02.398469 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:02.457140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70862269-cd89-4e66-aee3-95d1764268cd None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:02.460588 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf71bf-d56b-4ba5-af3d-df6311c4f419 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:02.495422 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa21ba-7020-44ab-b013-1ecd6b9760c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:02.496081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa21ba-7020-44ab-b013-1ecd6b9760c6 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:02.496364 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa21ba-7020-44ab-b013-1ecd6b9760c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:02.496638 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa21ba-7020-44ab-b013-1ecd6b9760c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:02.503928 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa21ba-7020-44ab-b013-1ecd6b9760c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:02.504409 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7360b-7c54-462f-b9e6-1fca545f7133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:02.505281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7360b-7c54-462f-b9e6-1fca545f7133 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:02.505385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa21ba-7020-44ab-b013-1ecd6b9760c6 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:02.506081 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7360b-7c54-462f-b9e6-1fca545f7133 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:02.506596 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7360b-7c54-462f-b9e6-1fca545f7133 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:02.514845 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7360b-7c54-462f-b9e6-1fca545f7133 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:02.516508 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7360b-7c54-462f-b9e6-1fca545f7133 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:02.516914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-3baa21ba-7020-44ab-b013-1ecd6b9760c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:02.516914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:02.516914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:02.516914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:02.516914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:02.516914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:02.516914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:02.516914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:02.516914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:02.516914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:02.516914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:02.516914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:02.516914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:02.516914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:02.516914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:02.517790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:02.517790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:02.517790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:02.517790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:02.517790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:02.517790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:02.517790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:02.517790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:02.517790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:02.517790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:02.517790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:02.517790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:02.517790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:02.517790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:02.517790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:02.517790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:02.517790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:02.518750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:02.518750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:02.518750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:02.518750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:02.518750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:02.518750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:02.518750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:02.518750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:02.523401 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 265/535] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:02.524268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa21ba-7020-44ab-b013-1ecd6b9760c6 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:02.536760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-7ea7360b-7c54-462f-b9e6-1fca545f7133 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:02.536760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:02.536760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:02.536760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:02.536760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:02.536760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:02.536760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:02.536760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:02.536760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:02.536760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:02.536760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:02.536760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:02.536760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:02.536760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:02.536760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:02.538257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:02.538257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:02.538257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:02.538257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:02.538257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:02.538257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:02.538257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:02.538257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:02.538257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:02.538257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:02.538257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:02.538257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:02.538257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:02.538257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:02.538257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:02.538257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:02.538257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:02.539120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:02.539120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:02.539120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:02.539120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:02.539120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:02.539120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:02.539120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:02.539120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:02.541957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 271/536] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:02.542961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7360b-7c54-462f-b9e6-1fca545f7133 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:02.641190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa21ba-7020-44ab-b013-1ecd6b9760c6 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:02.660585 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7360b-7c54-462f-b9e6-1fca545f7133 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:02.667288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:02.684959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c4bce-7068-4463-bfdc-115d93c3016f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:02.685950 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c4bce-7068-4463-bfdc-115d93c3016f None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:02.686047 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c4bce-7068-4463-bfdc-115d93c3016f None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:02.686223 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:02.686320 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c4bce-7068-4463-bfdc-115d93c3016f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:02.687132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c4bce-7068-4463-bfdc-115d93c3016f None admin] RBAC: Adding query filter params (role.id=391becda2bf04438b8d1763ab99acf1b, scope.project.id=4227a5b804c0414c99d407bb1415cf49, user.id=4abd1512ad224da896c6f2257f9602e3) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:02.687752 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c4bce-7068-4463-bfdc-115d93c3016f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:02.689428 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c4bce-7068-4463-bfdc-115d93c3016f None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:02.701921 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 266/537] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:47:02 2022] GET /identity/v3/role_assignments?user.id=4abd1512ad224da896c6f2257f9602e3&scope.project.id=4227a5b804c0414c99d407bb1415cf49&role.id=391becda2bf04438b8d1763ab99acf1b => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:02.701921 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c4bce-7068-4463-bfdc-115d93c3016f None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:02.706273 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786b2b0-7e4a-43d7-be84-0b1170dc1be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:02.707329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786b2b0-7e4a-43d7-be84-0b1170dc1be0 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:02.707806 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786b2b0-7e4a-43d7-be84-0b1170dc1be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:02.708272 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786b2b0-7e4a-43d7-be84-0b1170dc1be0 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:02.709220 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786b2b0-7e4a-43d7-be84-0b1170dc1be0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:02.709749 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786b2b0-7e4a-43d7-be84-0b1170dc1be0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:02.717131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786b2b0-7e4a-43d7-be84-0b1170dc1be0 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:02.741930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 272/538] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:02.748343 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786b2b0-7e4a-43d7-be84-0b1170dc1be0 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:02.830819 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c4bce-7068-4463-bfdc-115d93c3016f None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:02.877752 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977aab6-9972-4849-94aa-c3812662d245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:02.878668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977aab6-9972-4849-94aa-c3812662d245 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:02.878951 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977aab6-9972-4849-94aa-c3812662d245 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:02.879374 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977aab6-9972-4849-94aa-c3812662d245 None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:02.880505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977aab6-9972-4849-94aa-c3812662d245 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:02.880928 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977aab6-9972-4849-94aa-c3812662d245 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:02.883423 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977aab6-9972-4849-94aa-c3812662d245 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:02.887255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786b2b0-7e4a-43d7-be84-0b1170dc1be0 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:02.900265 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 267/539] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:02.901856 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ba3d6-feed-4561-b762-5647c81e91b4 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:02.902165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ba3d6-feed-4561-b762-5647c81e91b4 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:02.902605 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ba3d6-feed-4561-b762-5647c81e91b4 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:02.904561 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 268/540] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:02.907558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ba3d6-feed-4561-b762-5647c81e91b4 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:02.931692 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2fff63-a9df-4d38-b6df-8554a85ef7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:02.932246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fff63-a9df-4d38-b6df-8554a85ef7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:02.932533 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fff63-a9df-4d38-b6df-8554a85ef7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:02.932813 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2fff63-a9df-4d38-b6df-8554a85ef7d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:02.933610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2fff63-a9df-4d38-b6df-8554a85ef7d7 None admin] RBAC: Adding query filter params (role.id=391becda2bf04438b8d1763ab99acf1b, scope.project.id=4227a5b804c0414c99d407bb1415cf49, user.id=6e39392b79bc439780415db7c8206098) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:02.933949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2fff63-a9df-4d38-b6df-8554a85ef7d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:02.936971 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2fff63-a9df-4d38-b6df-8554a85ef7d7 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:02.945937 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 273/541] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:47:02 2022] GET /identity/v3/role_assignments?user.id=6e39392b79bc439780415db7c8206098&scope.project.id=4227a5b804c0414c99d407bb1415cf49&role.id=391becda2bf04438b8d1763ab99acf1b => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:02.945937 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970527c-9129-435f-b4c3-d767c65a07a8 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:02.945937 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970527c-9129-435f-b4c3-d767c65a07a8 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:02.945937 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970527c-9129-435f-b4c3-d767c65a07a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:02.965593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-1970527c-9129-435f-b4c3-d767c65a07a8 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:02.966309 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:02.967190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1970527c-9129-435f-b4c3-d767c65a07a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:02.983954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:03.039210 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ba3d6-feed-4561-b762-5647c81e91b4 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:03.071703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 274/542] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:47:03.095698 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970527c-9129-435f-b4c3-d767c65a07a8 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:03.135281 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8df2e-ce73-4261-ab6c-ae572110cf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:03.135918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8df2e-ce73-4261-ab6c-ae572110cf48 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:03.136209 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8df2e-ce73-4261-ab6c-ae572110cf48 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:03.136488 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8df2e-ce73-4261-ab6c-ae572110cf48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:03.137296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8df2e-ce73-4261-ab6c-ae572110cf48 None admin] RBAC: Adding query filter params (role.id=391becda2bf04438b8d1763ab99acf1b, scope.project.id=4227a5b804c0414c99d407bb1415cf49, user.id=ddf6f4d84bb34d2f810a0f6cec36da4d) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:03.137613 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8df2e-ce73-4261-ab6c-ae572110cf48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:03.139078 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8df2e-ce73-4261-ab6c-ae572110cf48 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:03.146278 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 269/543] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:47:02 2022] GET /identity/v3/role_assignments?user.id=ddf6f4d84bb34d2f810a0f6cec36da4d&scope.project.id=4227a5b804c0414c99d407bb1415cf49&role.id=391becda2bf04438b8d1763ab99acf1b => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:03.226594 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970527c-9129-435f-b4c3-d767c65a07a8 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:03.263625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aaa1a0-1124-4f35-a0e2-dab63094defa None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:03.263959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aaa1a0-1124-4f35-a0e2-dab63094defa None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:03.264183 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aaa1a0-1124-4f35-a0e2-dab63094defa None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:03.265690 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 270/544] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:47:03.273553 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df039ff-49a3-42e8-9e73-914f1639f822 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:03.274217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df039ff-49a3-42e8-9e73-914f1639f822 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:03.274718 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df039ff-49a3-42e8-9e73-914f1639f822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:03.277135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57832d3e-db4d-4d38-9a31-77c5078ee8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:03.277783 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57832d3e-db4d-4d38-9a31-77c5078ee8b1 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:03.278103 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57832d3e-db4d-4d38-9a31-77c5078ee8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:03.278384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57832d3e-db4d-4d38-9a31-77c5078ee8b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:03.293589 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57832d3e-db4d-4d38-9a31-77c5078ee8b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:47:03.297094 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-2df039ff-49a3-42e8-9e73-914f1639f822 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:03.298576 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-2df039ff-49a3-42e8-9e73-914f1639f822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:03.298773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-57832d3e-db4d-4d38-9a31-77c5078ee8b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:03.298773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:03.298773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:03.298773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:03.298773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:03.298773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:03.298773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:03.298773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:03.298773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:03.298773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:03.298773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:03.298773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:03.298773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:47:03.298773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:47:03.298773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:47:03.298773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:47:03.299658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:47:03.299658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:47:03.299658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:47:03.299658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:47:03.299658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:03.299658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:03.299658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:03.299658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:03.299658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:03.299658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:03.299658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:03.299658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:03.299658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:03.299658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:03.299658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:03.299658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:03.300873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:03.300873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:03.300873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:03.300873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:03.300873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:47:03.300873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:47:03.300873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:47:03.300873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:47:03.300873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:47:03.300873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:47:03.300873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:03.300873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:03.300873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 275/545] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 2 08:47:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:03.307418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57832d3e-db4d-4d38-9a31-77c5078ee8b1 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:03.324818 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:03.456274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 271/546] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:47:03.466180 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57832d3e-db4d-4d38-9a31-77c5078ee8b1 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:03.500638 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df039ff-49a3-42e8-9e73-914f1639f822 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:03.514694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98270dfa-b122-47a0-8892-68fe5eea7c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:03.515398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98270dfa-b122-47a0-8892-68fe5eea7c8b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:03.515686 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98270dfa-b122-47a0-8892-68fe5eea7c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:03.515964 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98270dfa-b122-47a0-8892-68fe5eea7c8b None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:03.516702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98270dfa-b122-47a0-8892-68fe5eea7c8b None admin] RBAC: Adding query filter params (name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:03.517010 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98270dfa-b122-47a0-8892-68fe5eea7c8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:03.518515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98270dfa-b122-47a0-8892-68fe5eea7c8b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:03.537838 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 276/547] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 2 08:47:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:03.540018 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98270dfa-b122-47a0-8892-68fe5eea7c8b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:03.571507 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:03.656860 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df039ff-49a3-42e8-9e73-914f1639f822 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:03.702362 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:03.741835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71022f8c-718e-4121-9ae1-065daf593032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:03.742502 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71022f8c-718e-4121-9ae1-065daf593032 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:03.742775 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71022f8c-718e-4121-9ae1-065daf593032 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:03.743043 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71022f8c-718e-4121-9ae1-065daf593032 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:03.763015 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71022f8c-718e-4121-9ae1-065daf593032 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:47:03.764351 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98270dfa-b122-47a0-8892-68fe5eea7c8b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:03.768790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-71022f8c-718e-4121-9ae1-065daf593032 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:03.768790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:03.768790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:03.768790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:03.768790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:03.768790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:03.768790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:03.768790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:03.768790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:03.768790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:03.768790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:03.768790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:03.768790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:47:03.768790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:47:03.768790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:47:03.768790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:47:03.769807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:47:03.769807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:47:03.769807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:47:03.769807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:47:03.769807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:03.769807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:03.769807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:03.769807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:03.769807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:03.769807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:03.769807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:03.769807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:03.769807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:03.769807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:03.769807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:03.769807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:03.770671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:03.770671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:03.770671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:03.770671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:03.770671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:47:03.770671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:47:03.770671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:47:03.770671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:47:03.770671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:47:03.770671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:47:03.770671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:03.770671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:03.778429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 272/548] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 2 08:47:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:03.779689 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71022f8c-718e-4121-9ae1-065daf593032 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:03.805574 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab583ed-4683-45fe-985d-19e54d41c617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:03.806596 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab583ed-4683-45fe-985d-19e54d41c617 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:03.807027 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab583ed-4683-45fe-985d-19e54d41c617 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:03.807350 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab583ed-4683-45fe-985d-19e54d41c617 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:03.820730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab583ed-4683-45fe-985d-19e54d41c617 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:03.822363 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab583ed-4683-45fe-985d-19e54d41c617 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:03.840301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-aab583ed-4683-45fe-985d-19e54d41c617 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:03.840301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:03.840301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:03.840301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:03.840301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:03.840301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:03.840301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:03.840301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:03.840301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:03.840301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:03.840301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:03.840301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:03.840301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:03.840301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:03.840301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:03.841415 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:03.841415 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:03.841415 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:03.841415 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:03.841415 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:03.841415 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:03.841415 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:03.841415 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:03.841415 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:03.841415 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:03.841415 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:03.841415 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:03.841415 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:03.841415 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:03.841415 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:03.841415 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:03.841415 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:03.849450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:03.849450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:03.849450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:03.849450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:03.849450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 277/549] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:03.849450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab583ed-4683-45fe-985d-19e54d41c617 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:03.981324 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab583ed-4683-45fe-985d-19e54d41c617 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:03.987106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:03.996742 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71022f8c-718e-4121-9ae1-065daf593032 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:04.050389 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:04.063341 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bffca6f-587d-4af7-a140-1618f15e1305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:04.063970 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bffca6f-587d-4af7-a140-1618f15e1305 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:04.064180 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759faccb-23dd-4c6a-881e-723c4b602c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:04.064300 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bffca6f-587d-4af7-a140-1618f15e1305 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:04.064511 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bffca6f-587d-4af7-a140-1618f15e1305 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:04.064794 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759faccb-23dd-4c6a-881e-723c4b602c9c None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:04.065074 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759faccb-23dd-4c6a-881e-723c4b602c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:04.065244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bffca6f-587d-4af7-a140-1618f15e1305 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:04.065338 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759faccb-23dd-4c6a-881e-723c4b602c9c None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:04.065507 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bffca6f-587d-4af7-a140-1618f15e1305 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:04.066069 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759faccb-23dd-4c6a-881e-723c4b602c9c None admin] RBAC: Adding query filter params (name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:04.066381 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759faccb-23dd-4c6a-881e-723c4b602c9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:04.066975 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bffca6f-587d-4af7-a140-1618f15e1305 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:04.067795 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759faccb-23dd-4c6a-881e-723c4b602c9c None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:04.078083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 273/550] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 2 08:47:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:04.084147 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759faccb-23dd-4c6a-881e-723c4b602c9c None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:04.091686 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 278/551] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:04.097595 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bffca6f-587d-4af7-a140-1618f15e1305 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:04.208595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759faccb-23dd-4c6a-881e-723c4b602c9c None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:04.247063 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1dfd4-b89f-4094-95ac-32d23ea8909f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:04.247708 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1dfd4-b89f-4094-95ac-32d23ea8909f None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:04.247985 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1dfd4-b89f-4094-95ac-32d23ea8909f None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:04.248261 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1dfd4-b89f-4094-95ac-32d23ea8909f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:04.261078 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1dfd4-b89f-4094-95ac-32d23ea8909f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:04.265913 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a1dfd4-b89f-4094-95ac-32d23ea8909f None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:04.283165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-48a1dfd4-b89f-4094-95ac-32d23ea8909f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:04.283165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:04.283165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:04.283165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:04.283165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:04.283165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:04.283165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:04.283165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:04.283165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:04.283165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:04.283165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:04.283165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:04.283165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:04.283165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:04.283165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:04.284466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:04.284466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:04.284466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:04.284466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:04.284466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:04.284466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:04.284466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:04.284466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:04.284466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:04.284466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:04.284466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:04.284466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:04.284466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:04.284466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:04.284466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:04.284466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:04.284466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:04.285358 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:04.285358 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:04.285358 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:04.285358 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:04.285358 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 274/552] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:04.293906 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1dfd4-b89f-4094-95ac-32d23ea8909f None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:04.306956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:04.340144 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bffca6f-587d-4af7-a140-1618f15e1305 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:04.402860 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a1dfd4-b89f-4094-95ac-32d23ea8909f None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:04.440345 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58de86-74b3-41b4-ad04-ebcd866145ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:04.441291 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58de86-74b3-41b4-ad04-ebcd866145ec None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:04.441570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58de86-74b3-41b4-ad04-ebcd866145ec None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:04.441909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58de86-74b3-41b4-ad04-ebcd866145ec None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:04.442614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58de86-74b3-41b4-ad04-ebcd866145ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:04.442916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58de86-74b3-41b4-ad04-ebcd866145ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:04.444333 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58de86-74b3-41b4-ad04-ebcd866145ec None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:04.456943 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 275/553] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:04.462314 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58de86-74b3-41b4-ad04-ebcd866145ec None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:04.470584 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b5dea-bf3f-4045-bc00-3f93269469ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:04.471254 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b5dea-bf3f-4045-bc00-3f93269469ac None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:04.471542 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b5dea-bf3f-4045-bc00-3f93269469ac None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:04.471823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b5dea-bf3f-4045-bc00-3f93269469ac None admin] PATH_INFO: `/v3/users/nova` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:04.502725 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b5dea-bf3f-4045-bc00-3f93269469ac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:04.504215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b5dea-bf3f-4045-bc00-3f93269469ac None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:04.539719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:04.540136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-079b5dea-bf3f-4045-bc00-3f93269469ac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:47:04.540136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:04.540136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:04.540136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:04.540136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:04.540136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:04.540136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:04.540136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:04.540136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:04.540136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:04.540136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:04.540136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:04.540136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:04.540136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:04.540136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:04.540136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:04.541050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:04.541050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:04.541050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:04.541050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:04.541050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:04.541050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:04.541050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:04.541050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:04.541050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:04.541050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:04.541050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:04.541050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:04.541050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:04.541050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:04.541050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:04.541050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:04.541050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:04.541050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:04.542407 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:04.542407 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:04.542407 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:04.542407 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:04.542407 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:04.542407 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:04.542407 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:04.542407 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:04.542407 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:47:04.542407 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:04.542407 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 279/554] 10.222.0.46 () {56 vars in 1093 bytes} [Tue Aug 2 08:47:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 445 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:04.550851 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b5dea-bf3f-4045-bc00-3f93269469ac None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:04.571760 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58de86-74b3-41b4-ad04-ebcd866145ec None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:04.611053 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef899e-0cd0-4371-ac17-85b9db23da2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:04.612091 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef899e-0cd0-4371-ac17-85b9db23da2d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:04.612371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef899e-0cd0-4371-ac17-85b9db23da2d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:04.612648 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef899e-0cd0-4371-ac17-85b9db23da2d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:04.621728 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef899e-0cd0-4371-ac17-85b9db23da2d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:04.623283 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef899e-0cd0-4371-ac17-85b9db23da2d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:04.636145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-57ef899e-0cd0-4371-ac17-85b9db23da2d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 08:47:04.636145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:04.636145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:04.636145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:04.636145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:04.636145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:04.636145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:04.636145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:04.636145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:04.636145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:04.636145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:04.636145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:04.636145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:04.636145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:04.636145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:04.636145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:04.638625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:04.638625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:04.638625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:04.638625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:04.638625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:04.638625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:04.638625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:04.638625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:04.638625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:04.638625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:04.638625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:04.638625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:04.638625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:04.638625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:04.638625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:04.638625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:04.638625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:04.638625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:04.641258 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:04.641258 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:04.641258 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:04.641258 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:04.641258 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:04.641258 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:04.641258 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:04.641258 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:04.641258 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 08:47:04.641258 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:04.641258 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 276/555] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:47:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:04.648276 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef899e-0cd0-4371-ac17-85b9db23da2d None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:04.719826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:04.762383 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef899e-0cd0-4371-ac17-85b9db23da2d None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:04.806269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:04.812023 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325268ed-0f8a-4109-a8ce-c8a6e69c29d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:04.812748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325268ed-0f8a-4109-a8ce-c8a6e69c29d2 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:04.813033 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325268ed-0f8a-4109-a8ce-c8a6e69c29d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:04.815279 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325268ed-0f8a-4109-a8ce-c8a6e69c29d2 None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:04.816110 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325268ed-0f8a-4109-a8ce-c8a6e69c29d2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:04.816423 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325268ed-0f8a-4109-a8ce-c8a6e69c29d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:04.817869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325268ed-0f8a-4109-a8ce-c8a6e69c29d2 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:04.853871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 277/556] 10.222.0.46 () {56 vars in 1135 bytes} [Tue Aug 2 08:47:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:04.856671 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b5dea-bf3f-4045-bc00-3f93269469ac None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:04.859064 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325268ed-0f8a-4109-a8ce-c8a6e69c29d2 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:04.924734 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c9996-fac4-40a2-bd80-61cc67b0866f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:04.925363 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c9996-fac4-40a2-bd80-61cc67b0866f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:04.925647 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c9996-fac4-40a2-bd80-61cc67b0866f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:04.925942 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c9996-fac4-40a2-bd80-61cc67b0866f None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:04.926815 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c9996-fac4-40a2-bd80-61cc67b0866f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:04.927126 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c9996-fac4-40a2-bd80-61cc67b0866f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:04.929478 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c9996-fac4-40a2-bd80-61cc67b0866f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:04.959252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 280/557] 10.222.0.46 () {56 vars in 1133 bytes} [Tue Aug 2 08:47:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 408 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:04.964387 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c9996-fac4-40a2-bd80-61cc67b0866f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:05.021660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325268ed-0f8a-4109-a8ce-c8a6e69c29d2 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:05.058933 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d02211-d720-4940-9b94-570a19443dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:05.059908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d02211-d720-4940-9b94-570a19443dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:05.060372 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d02211-d720-4940-9b94-570a19443dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:05.060799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d02211-d720-4940-9b94-570a19443dd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:05.115105 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c9996-fac4-40a2-bd80-61cc67b0866f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:05.116819 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:05.122463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:05.136343 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d02211-d720-4940-9b94-570a19443dd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:05.137790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d02211-d720-4940-9b94-570a19443dd9 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:05.159576 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02dd71a-d3b5-4b69-afe6-73c17f37e68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:05.160189 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dd71a-d3b5-4b69-afe6-73c17f37e68d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:05.160522 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dd71a-d3b5-4b69-afe6-73c17f37e68d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:05.160804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dd71a-d3b5-4b69-afe6-73c17f37e68d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:05.171535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02dd71a-d3b5-4b69-afe6-73c17f37e68d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:05.172998 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02dd71a-d3b5-4b69-afe6-73c17f37e68d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:05.188707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-b02dd71a-d3b5-4b69-afe6-73c17f37e68d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:05.188707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:05.188707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:05.188707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:05.188707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:05.188707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:05.188707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:05.188707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:05.188707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:05.188707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:05.188707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:05.188707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:05.188707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:05.188707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:05.188707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:05.189862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:05.189862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:05.189862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:05.189862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:05.189862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:05.189862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:05.189862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:05.189862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:05.189862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:05.189862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:05.189862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:05.189862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:05.189862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:05.189862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:05.189862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:05.189862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:05.189862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:05.190792 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:05.190792 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:05.190792 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:05.190792 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:05.190792 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 281/558] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:05.191535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7357e41-9b36-445b-a4df-1aaabebbfced None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:05.191694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7357e41-9b36-445b-a4df-1aaabebbfced None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:05.193280 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7357e41-9b36-445b-a4df-1aaabebbfced None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:05.193509 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 282/559] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:05.195764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7357e41-9b36-445b-a4df-1aaabebbfced None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:05.248549 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 278/560] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:47:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:47:05.249829 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6ed88-9f41-42d2-8974-033d236d735a None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:05.250202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6ed88-9f41-42d2-8974-033d236d735a None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:05.250485 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6ed88-9f41-42d2-8974-033d236d735a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:05.270102 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-12b6ed88-9f41-42d2-8974-033d236d735a None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:05.271301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-12b6ed88-9f41-42d2-8974-033d236d735a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:05.314374 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7357e41-9b36-445b-a4df-1aaabebbfced None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:05.344633 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 279/561] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:47:05.346002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6ed88-9f41-42d2-8974-033d236d735a None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:05.350904 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a75382b-ffe8-4e11-b2f4-e36676a13d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:05.351630 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75382b-ffe8-4e11-b2f4-e36676a13d21 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:05.351984 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75382b-ffe8-4e11-b2f4-e36676a13d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:05.352331 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75382b-ffe8-4e11-b2f4-e36676a13d21 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:05.353095 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a75382b-ffe8-4e11-b2f4-e36676a13d21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:05.353468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a75382b-ffe8-4e11-b2f4-e36676a13d21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:05.354968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a75382b-ffe8-4e11-b2f4-e36676a13d21 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:05.369013 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 283/562] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:05.370380 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d262d2-4f4b-41ce-8ab5-1be4299c0cdd None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:05.370742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d262d2-4f4b-41ce-8ab5-1be4299c0cdd None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:05.371086 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d262d2-4f4b-41ce-8ab5-1be4299c0cdd None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:05.372608 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 284/563] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:05.375579 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d262d2-4f4b-41ce-8ab5-1be4299c0cdd None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:05.379304 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:05.393301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:05.465421 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6ed88-9f41-42d2-8974-033d236d735a None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:05.490385 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d262d2-4f4b-41ce-8ab5-1be4299c0cdd None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:05.508133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010eca2-bd18-42ce-9b3e-7534449efe86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:05.508133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010eca2-bd18-42ce-9b3e-7534449efe86 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:05.508133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010eca2-bd18-42ce-9b3e-7534449efe86 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:05.508133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010eca2-bd18-42ce-9b3e-7534449efe86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:05.518479 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010eca2-bd18-42ce-9b3e-7534449efe86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:05.519960 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010eca2-bd18-42ce-9b3e-7534449efe86 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:05.530602 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd22693-83f7-41c6-a3e4-bc48137a3634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:05.531244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd22693-83f7-41c6-a3e4-bc48137a3634 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:05.531527 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd22693-83f7-41c6-a3e4-bc48137a3634 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:05.531805 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd22693-83f7-41c6-a3e4-bc48137a3634 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:05.536081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-8010eca2-bd18-42ce-9b3e-7534449efe86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:05.536081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:05.536081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:05.536081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:05.536081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:05.536081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:05.536081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:05.536081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:05.536081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:05.536081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:05.536081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:05.536081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:05.536081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:05.536081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:05.536081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:05.536934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:05.536934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:05.536934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:05.536934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:05.536934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:05.536934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:05.536934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:05.536934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:05.536934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:05.536934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:05.536934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:05.536934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:05.536934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:05.536934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:05.536934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:05.536934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:05.536934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:05.538507 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:05.538507 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:05.538507 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:05.538507 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:05.538507 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 280/564] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:05.539252 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010eca2-bd18-42ce-9b3e-7534449efe86 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:05.544455 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd22693-83f7-41c6-a3e4-bc48137a3634 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:05.545920 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd22693-83f7-41c6-a3e4-bc48137a3634 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:05.558737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-9bd22693-83f7-41c6-a3e4-bc48137a3634 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:05.558737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:05.558737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:05.558737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:05.558737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:05.558737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:05.558737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:05.558737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:05.558737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:05.558737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:05.558737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:05.558737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:05.558737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:05.558737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:05.558737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:05.560419 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:05.560419 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:05.560419 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:05.560419 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:05.560419 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:05.560419 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:05.560419 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:05.560419 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:05.560419 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:05.560419 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:05.560419 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:05.560419 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:05.560419 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:05.560419 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:05.560419 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:05.560419 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:05.560419 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:05.562236 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:05.562236 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:05.562236 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:05.562236 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:05.562236 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:05.562236 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:05.562236 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:05.562236 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:05.564658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 285/565] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:05.565933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cddc2-ae26-4a2c-bc93-0d9262e37830 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:05.566227 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cddc2-ae26-4a2c-bc93-0d9262e37830 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:05.566642 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12cddc2-ae26-4a2c-bc93-0d9262e37830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:05.584833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-e12cddc2-ae26-4a2c-bc93-0d9262e37830 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:05.586061 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e12cddc2-ae26-4a2c-bc93-0d9262e37830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:05.617768 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:05.647826 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010eca2-bd18-42ce-9b3e-7534449efe86 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:05.652618 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 286/566] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:47:05.653262 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cddc2-ae26-4a2c-bc93-0d9262e37830 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:05.664895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:05.681614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5fe1e-4293-49f6-9cc8-06a5bfe7ea49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:05.682396 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5fe1e-4293-49f6-9cc8-06a5bfe7ea49 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:05.682677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5fe1e-4293-49f6-9cc8-06a5bfe7ea49 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:05.682954 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f5fe1e-4293-49f6-9cc8-06a5bfe7ea49 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:05.693124 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f5fe1e-4293-49f6-9cc8-06a5bfe7ea49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:47:05.698816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-32f5fe1e-4293-49f6-9cc8-06a5bfe7ea49 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:05.698816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:05.698816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:05.698816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:05.698816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:05.698816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:05.698816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:05.698816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:05.698816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:05.698816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:05.698816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:05.698816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:05.698816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:47:05.698816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:47:05.698816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:47:05.698816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:47:05.699816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:47:05.699816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:47:05.699816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:47:05.699816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:47:05.699816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:05.699816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:05.699816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:05.699816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:05.699816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:05.699816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:05.699816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:05.699816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:05.699816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:05.699816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:05.699816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:05.699816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:05.700661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:05.700661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:05.700661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:05.700661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:05.700661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:47:05.700661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:47:05.700661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:47:05.700661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:47:05.700661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:47:05.700661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:47:05.700661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:05.700661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:05.700661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 281/567] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 2 08:47:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:05.701821 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5fe1e-4293-49f6-9cc8-06a5bfe7ea49 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:05.762700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12cddc2-ae26-4a2c-bc93-0d9262e37830 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:05.798012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe990b-e183-48be-ada5-e7c40390b853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:05.798635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe990b-e183-48be-ada5-e7c40390b853 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:05.798918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe990b-e183-48be-ada5-e7c40390b853 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:05.799192 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe990b-e183-48be-ada5-e7c40390b853 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:05.799884 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe990b-e183-48be-ada5-e7c40390b853 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:05.800190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe990b-e183-48be-ada5-e7c40390b853 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:05.801626 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe990b-e183-48be-ada5-e7c40390b853 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:05.807468 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f5fe1e-4293-49f6-9cc8-06a5bfe7ea49 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:05.813437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 287/568] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:05.814722 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f6662-1479-485d-b118-91684405032a None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:05.815014 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f6662-1479-485d-b118-91684405032a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:05.815290 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f6662-1479-485d-b118-91684405032a None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:05.816661 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 288/569] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:05.817250 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f6662-1479-485d-b118-91684405032a None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:05.841668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:05.846669 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca5584-b24c-4496-8a1e-ab12cb060e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:05.847285 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca5584-b24c-4496-8a1e-ab12cb060e84 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:05.847566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca5584-b24c-4496-8a1e-ab12cb060e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:05.847842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca5584-b24c-4496-8a1e-ab12cb060e84 None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:05.848586 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca5584-b24c-4496-8a1e-ab12cb060e84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:05.848894 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca5584-b24c-4496-8a1e-ab12cb060e84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:05.850943 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca5584-b24c-4496-8a1e-ab12cb060e84 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:05.862804 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 282/570] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:05.863314 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca5584-b24c-4496-8a1e-ab12cb060e84 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:05.938071 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:05.964643 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca5584-b24c-4496-8a1e-ab12cb060e84 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:05.977993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f6662-1479-485d-b118-91684405032a None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:05.997374 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fd590-5832-4310-a8dc-94e8ae3a8d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:05.998035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fd590-5832-4310-a8dc-94e8ae3a8d50 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:05.998317 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fd590-5832-4310-a8dc-94e8ae3a8d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:05.998594 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fd590-5832-4310-a8dc-94e8ae3a8d50 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:05.999306 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724fd590-5832-4310-a8dc-94e8ae3a8d50 None admin] RBAC: Adding query filter params (name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:05.999613 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724fd590-5832-4310-a8dc-94e8ae3a8d50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:06.001239 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724fd590-5832-4310-a8dc-94e8ae3a8d50 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:06.009314 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 283/571] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 2 08:47:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:06.010107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fd590-5832-4310-a8dc-94e8ae3a8d50 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:06.015668 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1f813b-d546-495d-bcf2-dee5663be013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:06.016302 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f813b-d546-495d-bcf2-dee5663be013 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:06.016583 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f813b-d546-495d-bcf2-dee5663be013 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:06.016857 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1f813b-d546-495d-bcf2-dee5663be013 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:06.027737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1f813b-d546-495d-bcf2-dee5663be013 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:47:06.032978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-5f1f813b-d546-495d-bcf2-dee5663be013 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:06.032978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:06.032978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:06.032978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:06.032978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:06.032978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:06.032978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:06.032978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:06.032978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:06.032978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:06.032978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:06.032978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:06.032978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:47:06.032978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:47:06.032978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:47:06.032978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:47:06.033969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:47:06.033969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:47:06.033969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:47:06.033969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:47:06.033969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:06.033969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:06.033969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:06.033969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:06.033969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:06.033969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:06.033969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:06.033969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:06.033969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:06.033969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:06.033969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:06.033969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:06.034935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:06.034935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:06.034935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:06.034935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:06.034935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:47:06.034935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:47:06.034935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:47:06.034935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:47:06.034935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:47:06.034935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:47:06.034935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:06.034935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:06.034935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 289/572] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 2 08:47:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:06.035657 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8462b-3adb-42f3-a9cd-b42d26d57a20 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:06.035657 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8462b-3adb-42f3-a9cd-b42d26d57a20 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:06.036385 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8462b-3adb-42f3-a9cd-b42d26d57a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:06.053023 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-75c8462b-3adb-42f3-a9cd-b42d26d57a20 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:06.054457 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-75c8462b-3adb-42f3-a9cd-b42d26d57a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:06.067688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:06.112444 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fd590-5832-4310-a8dc-94e8ae3a8d50 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:06.120552 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 290/573] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:47:06.121360 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8462b-3adb-42f3-a9cd-b42d26d57a20 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:06.159144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb5a82-cd49-4c9a-ba90-1ab221d8a85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:06.159916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb5a82-cd49-4c9a-ba90-1ab221d8a85a None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:06.160199 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb5a82-cd49-4c9a-ba90-1ab221d8a85a None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:06.160474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb5a82-cd49-4c9a-ba90-1ab221d8a85a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:06.168028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb5a82-cd49-4c9a-ba90-1ab221d8a85a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:06.169707 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb5a82-cd49-4c9a-ba90-1ab221d8a85a None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:06.181018 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:06.181602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-49cb5a82-cd49-4c9a-ba90-1ab221d8a85a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:06.181602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:06.181602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:06.181602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:06.181602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:06.181602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:06.181602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:06.181602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:06.181602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:06.181602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:06.181602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:06.181602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:06.181602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:06.181602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:06.181602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:06.182835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:06.182835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:06.182835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:06.182835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:06.182835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:06.182835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:06.182835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:06.182835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:06.182835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:06.182835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:06.182835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:06.182835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:06.182835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:06.182835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:06.182835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:06.182835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:06.182835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:06.184805 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:06.184805 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:06.184805 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:06.184805 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:06.184805 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:06.184805 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:06.184805 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:06.184805 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:06.186016 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 284/574] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:06.186853 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb5a82-cd49-4c9a-ba90-1ab221d8a85a None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:06.228978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8462b-3adb-42f3-a9cd-b42d26d57a20 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:06.261100 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1707d-e4c6-44e5-b796-d5d2e166e617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:06.261764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1707d-e4c6-44e5-b796-d5d2e166e617 None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:06.262083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1707d-e4c6-44e5-b796-d5d2e166e617 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:06.262363 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1707d-e4c6-44e5-b796-d5d2e166e617 None admin] PATH_INFO: `/v3/projects/4227a5b804c0414c99d407bb1415cf49/users/3191d804e8dc4b1a85f490319833ec07/roles/391becda2bf04438b8d1763ab99acf1b` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:06.276535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa1707d-e4c6-44e5-b796-d5d2e166e617 None admin] RBAC: Authorizing `identity:create_grant(project_id=4227a5b804c0414c99d407bb1415cf49, user_id=3191d804e8dc4b1a85f490319833ec07, role_id=391becda2bf04438b8d1763ab99acf1b)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:06.278003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa1707d-e4c6-44e5-b796-d5d2e166e617 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:06.285025 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb5a82-cd49-4c9a-ba90-1ab221d8a85a None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:06.298734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:06.317988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c275be9-00e7-44cb-b1c8-4512087c60a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:06.318665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c275be9-00e7-44cb-b1c8-4512087c60a5 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:06.318965 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c275be9-00e7-44cb-b1c8-4512087c60a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:06.319245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c275be9-00e7-44cb-b1c8-4512087c60a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:06.329547 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c275be9-00e7-44cb-b1c8-4512087c60a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:06.331011 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c275be9-00e7-44cb-b1c8-4512087c60a5 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:06.345337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-9c275be9-00e7-44cb-b1c8-4512087c60a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:06.345337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:06.345337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:06.345337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:06.345337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:06.345337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:06.345337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:06.345337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:06.345337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:06.345337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:06.345337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:06.345337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:06.345337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:06.345337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:06.345337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:06.346361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:06.346361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:06.346361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:06.346361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:06.346361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:06.346361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:06.346361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:06.346361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:06.346361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:06.346361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:06.346361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:06.346361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:06.346361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:06.346361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:06.346361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:06.346361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:06.346361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:06.347262 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:06.347262 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:06.347262 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:06.347262 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:06.347262 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 285/576] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:06.347262 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c275be9-00e7-44cb-b1c8-4512087c60a5 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:06.351945 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 291/576] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:47:06 2022] PUT /identity/v3/projects/4227a5b804c0414c99d407bb1415cf49/users/3191d804e8dc4b1a85f490319833ec07/roles/391becda2bf04438b8d1763ab99acf1b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:47:06.352658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1707d-e4c6-44e5-b796-d5d2e166e617 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:06.479253 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c275be9-00e7-44cb-b1c8-4512087c60a5 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:06.495197 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1707d-e4c6-44e5-b796-d5d2e166e617 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:06.499447 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:06.524424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e6e81-8495-4529-b2cf-4620fb47b570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:06.525058 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e6e81-8495-4529-b2cf-4620fb47b570 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:06.525340 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e6e81-8495-4529-b2cf-4620fb47b570 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:06.525615 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e6e81-8495-4529-b2cf-4620fb47b570 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:06.526376 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e6e81-8495-4529-b2cf-4620fb47b570 None admin] RBAC: Adding query filter params (name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:06.526697 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e6e81-8495-4529-b2cf-4620fb47b570 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:06.529208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e6e81-8495-4529-b2cf-4620fb47b570 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:06.539275 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 286/577] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 2 08:47:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:06.539937 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e6e81-8495-4529-b2cf-4620fb47b570 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:06.546197 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a0953-5097-4aec-8dca-0af1a54fa068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:06.546852 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a0953-5097-4aec-8dca-0af1a54fa068 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:06.547134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a0953-5097-4aec-8dca-0af1a54fa068 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:06.547401 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a0953-5097-4aec-8dca-0af1a54fa068 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:06.558259 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a0953-5097-4aec-8dca-0af1a54fa068 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:47:06.563492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-d60a0953-5097-4aec-8dca-0af1a54fa068 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:06.563492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:06.563492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:06.563492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:06.563492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:06.563492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:06.563492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:06.563492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:06.563492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:06.563492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:06.563492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:06.563492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:06.563492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:47:06.563492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:47:06.563492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:47:06.563492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:47:06.564609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:47:06.564609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:47:06.564609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:47:06.564609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:47:06.564609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:06.564609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:06.564609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:06.564609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:06.564609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:06.564609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:06.564609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:06.564609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:06.564609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:06.564609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:06.564609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:06.564609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:06.565477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:06.565477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:06.565477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:06.565477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:06.565477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:47:06.565477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:47:06.565477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:47:06.565477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:47:06.565477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:47:06.565477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:47:06.565477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:06.565477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:06.565477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 292/578] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 2 08:47:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:06.565477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a0953-5097-4aec-8dca-0af1a54fa068 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:06.577686 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:06.654434 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e6e81-8495-4529-b2cf-4620fb47b570 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:06.679600 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a0953-5097-4aec-8dca-0af1a54fa068 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:06.688797 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c172e71-d848-49ec-9405-6b39fa9cb275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:06.689537 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c172e71-d848-49ec-9405-6b39fa9cb275 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:06.689951 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c172e71-d848-49ec-9405-6b39fa9cb275 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:06.690355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c172e71-d848-49ec-9405-6b39fa9cb275 None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:06.691184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c172e71-d848-49ec-9405-6b39fa9cb275 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:06.691567 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c172e71-d848-49ec-9405-6b39fa9cb275 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:06.693060 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c172e71-d848-49ec-9405-6b39fa9cb275 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:06.711558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 287/579] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:06.711558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c172e71-d848-49ec-9405-6b39fa9cb275 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:06.722226 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274e0f7-5a6a-436b-bcc4-badce720e8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:06.722844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274e0f7-5a6a-436b-bcc4-badce720e8df None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:06.723128 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274e0f7-5a6a-436b-bcc4-badce720e8df None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:06.723421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274e0f7-5a6a-436b-bcc4-badce720e8df None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:06.724130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274e0f7-5a6a-436b-bcc4-badce720e8df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:06.724433 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274e0f7-5a6a-436b-bcc4-badce720e8df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:06.726350 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274e0f7-5a6a-436b-bcc4-badce720e8df None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:06.742871 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 293/580] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:06.743683 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274e0f7-5a6a-436b-bcc4-badce720e8df None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:06.771241 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:06.805037 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:06.818543 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c172e71-d848-49ec-9405-6b39fa9cb275 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:06.860797 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545328ea-26d9-49fd-8c2f-f4e694098b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:06.861441 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545328ea-26d9-49fd-8c2f-f4e694098b0c None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:06.861744 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545328ea-26d9-49fd-8c2f-f4e694098b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:06.862074 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545328ea-26d9-49fd-8c2f-f4e694098b0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:06.873779 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274e0f7-5a6a-436b-bcc4-badce720e8df None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:06.874994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545328ea-26d9-49fd-8c2f-f4e694098b0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:06.876971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545328ea-26d9-49fd-8c2f-f4e694098b0c None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:06.893641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-545328ea-26d9-49fd-8c2f-f4e694098b0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:06.893641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:06.893641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:06.893641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:06.893641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:06.893641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:06.893641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:06.893641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:06.893641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:06.893641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:06.893641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:06.893641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:06.893641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:06.893641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:06.893641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:06.894885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:06.894885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:06.894885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:06.894885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:06.894885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:06.894885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:06.894885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:06.894885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:06.894885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:06.894885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:06.894885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:06.894885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:06.894885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:06.894885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:06.894885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:06.894885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:06.894885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:06.895978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:06.895978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:06.895978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:06.895978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:06.896743 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 288/581] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:06.898284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545328ea-26d9-49fd-8c2f-f4e694098b0c None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:06.916969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a55fb-abda-4314-8c4f-c9c0dc410f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:06.917597 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a55fb-abda-4314-8c4f-c9c0dc410f1d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:06.917878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a55fb-abda-4314-8c4f-c9c0dc410f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:06.918196 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a55fb-abda-4314-8c4f-c9c0dc410f1d None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:06.918909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a55fb-abda-4314-8c4f-c9c0dc410f1d None admin] RBAC: Adding query filter params (name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:06.919218 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a55fb-abda-4314-8c4f-c9c0dc410f1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:06.920577 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a55fb-abda-4314-8c4f-c9c0dc410f1d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:06.936168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 294/582] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 2 08:47:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:06.936168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a55fb-abda-4314-8c4f-c9c0dc410f1d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:07.014042 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545328ea-26d9-49fd-8c2f-f4e694098b0c None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:07.030179 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:07.046063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a55fb-abda-4314-8c4f-c9c0dc410f1d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:07.048140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fd9db-56da-47fe-9eec-c5b300bc4146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:07.048759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fd9db-56da-47fe-9eec-c5b300bc4146 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:07.049038 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fd9db-56da-47fe-9eec-c5b300bc4146 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:07.049311 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fd9db-56da-47fe-9eec-c5b300bc4146 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:07.050125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0fd9db-56da-47fe-9eec-c5b300bc4146 None admin] RBAC: Adding query filter params (role.id=391becda2bf04438b8d1763ab99acf1b, scope.project.id=4227a5b804c0414c99d407bb1415cf49, user.id=ff92ef3b9da74ccab3fbc43f852e3ad0) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:07.050435 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0fd9db-56da-47fe-9eec-c5b300bc4146 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:07.052075 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0fd9db-56da-47fe-9eec-c5b300bc4146 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:07.058337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 289/583] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:47:06 2022] GET /identity/v3/role_assignments?user.id=ff92ef3b9da74ccab3fbc43f852e3ad0&scope.project.id=4227a5b804c0414c99d407bb1415cf49&role.id=391becda2bf04438b8d1763ab99acf1b => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:07.059111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fd9db-56da-47fe-9eec-c5b300bc4146 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:07.067700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:07.102649 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c78b0-93b7-4a81-92ff-b2e9cdd8a1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:07.103444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c78b0-93b7-4a81-92ff-b2e9cdd8a1be None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:07.103800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c78b0-93b7-4a81-92ff-b2e9cdd8a1be None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:07.104150 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5c78b0-93b7-4a81-92ff-b2e9cdd8a1be None admin] PATH_INFO: `/v3/users/glance` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:07.113023 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c78b0-93b7-4a81-92ff-b2e9cdd8a1be None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:07.114681 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5c78b0-93b7-4a81-92ff-b2e9cdd8a1be None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:07.128054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-5a5c78b0-93b7-4a81-92ff-b2e9cdd8a1be None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 02 08:47:07.128054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:07.128054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:07.128054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:07.128054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:07.128054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:07.128054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:07.128054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:07.128054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:07.128054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:07.128054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:07.128054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:07.128054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:07.128054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:07.128054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:07.128054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:07.129127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:07.129127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:07.129127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:07.129127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:07.129127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:07.129127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:07.129127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:07.129127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:07.129127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:07.129127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:07.129127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:07.129127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:07.129127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:07.129127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:07.129127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:07.129127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:07.129127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:07.129127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:07.130328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:07.130328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:07.130328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:07.130328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:07.130328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:07.130328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:07.130328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:07.130328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:07.130328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 02 08:47:07.130328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:07.130328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 295/584] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:47:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:07.131145 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c78b0-93b7-4a81-92ff-b2e9cdd8a1be None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:07.190737 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fd9db-56da-47fe-9eec-c5b300bc4146 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:07.231155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f65f5c-af99-4187-b6b9-fe326a27362d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:07.232104 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f65f5c-af99-4187-b6b9-fe326a27362d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:07.232577 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f65f5c-af99-4187-b6b9-fe326a27362d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:07.233041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f65f5c-af99-4187-b6b9-fe326a27362d None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:07.234112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f65f5c-af99-4187-b6b9-fe326a27362d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:07.234614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f65f5c-af99-4187-b6b9-fe326a27362d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:07.236202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f65f5c-af99-4187-b6b9-fe326a27362d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:07.247314 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5c78b0-93b7-4a81-92ff-b2e9cdd8a1be None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:07.252562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 290/585] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:07.253629 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f65f5c-af99-4187-b6b9-fe326a27362d None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:07.288180 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f99ad-4864-424b-b308-49aebf2632be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:07.288741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f99ad-4864-424b-b308-49aebf2632be None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:07.289062 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f99ad-4864-424b-b308-49aebf2632be None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:07.289348 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f99ad-4864-424b-b308-49aebf2632be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:07.300662 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820f99ad-4864-424b-b308-49aebf2632be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:07.302231 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820f99ad-4864-424b-b308-49aebf2632be None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:07.308979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:07.317941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-820f99ad-4864-424b-b308-49aebf2632be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:07.317941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:07.317941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:07.317941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:07.317941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:07.317941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:07.317941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:07.317941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:07.317941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:07.317941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:07.317941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:07.317941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:07.317941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:07.317941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:07.317941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:07.326255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:07.326255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:07.326255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:07.326255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:07.326255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:07.326255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:07.326255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:07.326255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:07.326255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:07.326255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:07.326255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:07.326255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:07.326255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:07.326255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:07.326255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:07.326255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:07.326255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:07.327169 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:07.327169 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:07.327169 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:07.327169 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:07.327169 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 296/586] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:07.327169 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f99ad-4864-424b-b308-49aebf2632be None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:07.338548 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:07.386997 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f65f5c-af99-4187-b6b9-fe326a27362d None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:07.431657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd757cc0-7d45-4c90-9d39-ec0b4f1ef94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:07.432393 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd757cc0-7d45-4c90-9d39-ec0b4f1ef94f None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:07.432751 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd757cc0-7d45-4c90-9d39-ec0b4f1ef94f None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:07.433106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd757cc0-7d45-4c90-9d39-ec0b4f1ef94f None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:07.434417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd757cc0-7d45-4c90-9d39-ec0b4f1ef94f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:07.434417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd757cc0-7d45-4c90-9d39-ec0b4f1ef94f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:07.435928 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd757cc0-7d45-4c90-9d39-ec0b4f1ef94f None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:07.447330 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f99ad-4864-424b-b308-49aebf2632be None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:07.455808 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 291/587] 10.222.0.46 () {56 vars in 1137 bytes} [Tue Aug 2 08:47:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:07.456886 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd757cc0-7d45-4c90-9d39-ec0b4f1ef94f None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:07.485749 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d94b6-ef4f-4426-8b28-02e0a83b76ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:07.486912 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d94b6-ef4f-4426-8b28-02e0a83b76ca None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:07.487503 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d94b6-ef4f-4426-8b28-02e0a83b76ca None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:07.487965 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12d94b6-ef4f-4426-8b28-02e0a83b76ca None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:07.498343 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12d94b6-ef4f-4426-8b28-02e0a83b76ca None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:07.500631 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12d94b6-ef4f-4426-8b28-02e0a83b76ca None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:07.514028 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-b12d94b6-ef4f-4426-8b28-02e0a83b76ca None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 08:47:07.514028 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:07.514028 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:07.514028 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:07.514028 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:07.514028 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:07.514028 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:07.514028 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:07.514028 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:07.514028 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:07.514028 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:07.514028 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:07.514028 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:07.514028 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:07.514028 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:07.514028 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:07.515003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:07.515003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:07.515003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:07.515003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:07.515003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:07.515003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:07.515003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:07.515003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:07.515003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:07.515003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:07.515003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:07.515003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:07.515003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:07.515003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:07.515003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:07.515003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:07.515003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:07.515003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:07.516005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:07.516005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:07.516005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:07.516005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:07.516005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:07.516005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:07.516005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:07.516005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:07.516005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 08:47:07.516005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:07.522954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 297/588] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:47:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:07.524210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d94b6-ef4f-4426-8b28-02e0a83b76ca None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:07.568318 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd757cc0-7d45-4c90-9d39-ec0b4f1ef94f None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:07.592883 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:07.594968 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:07.609916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca9166-f8d1-4390-bcf4-c100e1ebd238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:07.609916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca9166-f8d1-4390-bcf4-c100e1ebd238 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:07.609916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca9166-f8d1-4390-bcf4-c100e1ebd238 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:07.609916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca9166-f8d1-4390-bcf4-c100e1ebd238 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:07.610288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca9166-f8d1-4390-bcf4-c100e1ebd238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:07.610288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca9166-f8d1-4390-bcf4-c100e1ebd238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:07.610288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baca9166-f8d1-4390-bcf4-c100e1ebd238 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:07.622283 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 292/589] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:07.627292 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca9166-f8d1-4390-bcf4-c100e1ebd238 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:07.651756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12d94b6-ef4f-4426-8b28-02e0a83b76ca None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:07.690782 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deea7df-7df6-4eaa-9e01-c015db2b5686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:07.691427 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deea7df-7df6-4eaa-9e01-c015db2b5686 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:07.691712 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deea7df-7df6-4eaa-9e01-c015db2b5686 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:07.691995 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deea7df-7df6-4eaa-9e01-c015db2b5686 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:07.703538 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deea7df-7df6-4eaa-9e01-c015db2b5686 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:07.705066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deea7df-7df6-4eaa-9e01-c015db2b5686 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:07.719617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-8deea7df-7df6-4eaa-9e01-c015db2b5686 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:07.719617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:07.719617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:07.719617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:07.719617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:07.719617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:07.719617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:07.719617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:07.719617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:07.719617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:07.719617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:07.719617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:07.719617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:07.719617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:07.719617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:07.720543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:07.720543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:07.720543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:07.720543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:07.720543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:07.720543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:07.720543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:07.720543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:07.720543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:07.720543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:07.720543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:07.720543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:07.720543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:07.720543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:07.720543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:07.720543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:07.720543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:07.721445 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:07.721445 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:07.721445 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:07.721445 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:07.725400 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 298/590] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:07.726311 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deea7df-7df6-4eaa-9e01-c015db2b5686 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:07.732013 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baca9166-f8d1-4390-bcf4-c100e1ebd238 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:07.767491 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632f3e2-063a-450d-aa0a-d918e60ecce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:07.768114 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632f3e2-063a-450d-aa0a-d918e60ecce0 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:07.768404 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632f3e2-063a-450d-aa0a-d918e60ecce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:07.768688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3632f3e2-063a-450d-aa0a-d918e60ecce0 None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:07.769484 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632f3e2-063a-450d-aa0a-d918e60ecce0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:07.769798 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632f3e2-063a-450d-aa0a-d918e60ecce0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:07.771289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3632f3e2-063a-450d-aa0a-d918e60ecce0 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:07.790340 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 293/591] 10.222.0.46 () {56 vars in 1137 bytes} [Tue Aug 2 08:47:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:07.791252 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632f3e2-063a-450d-aa0a-d918e60ecce0 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:07.830556 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deea7df-7df6-4eaa-9e01-c015db2b5686 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:07.848622 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:07.849415 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:07.870980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee738c83-5589-4829-826a-4b397ea8babf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:07.871846 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee738c83-5589-4829-826a-4b397ea8babf None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:07.872680 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee738c83-5589-4829-826a-4b397ea8babf None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:07.872967 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee738c83-5589-4829-826a-4b397ea8babf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:07.883578 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee738c83-5589-4829-826a-4b397ea8babf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:07.885737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee738c83-5589-4829-826a-4b397ea8babf None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:07.901252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-ee738c83-5589-4829-826a-4b397ea8babf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 08:47:07.901252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:07.901252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:07.901252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:07.901252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:07.901252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:07.901252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:07.901252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:07.901252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:07.901252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:07.901252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:07.901252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:07.901252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:07.901252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:07.901252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:07.901252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:07.902268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:07.902268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:07.902268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:07.902268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:07.902268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:07.902268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:07.902268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:07.902268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:07.902268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:07.902268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:07.902268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:07.902268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:07.902268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:07.902268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:07.902268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:07.902268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:07.902268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:07.902268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:07.903230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:07.903230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:07.903230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:07.903230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:07.903230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:07.903230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:07.903230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:07.903230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:07.903230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 08:47:07.903230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:07.903230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 299/592] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 2 08:47:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:07.904229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee738c83-5589-4829-826a-4b397ea8babf None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:07.911715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3632f3e2-063a-450d-aa0a-d918e60ecce0 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:07.961465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3145e7-58cf-423a-a876-d8869e5a11a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:07.962170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3145e7-58cf-423a-a876-d8869e5a11a8 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:07.962459 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3145e7-58cf-423a-a876-d8869e5a11a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:07.962740 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3145e7-58cf-423a-a876-d8869e5a11a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:07.963584 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3145e7-58cf-423a-a876-d8869e5a11a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:07.963898 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3145e7-58cf-423a-a876-d8869e5a11a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:07.965389 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3145e7-58cf-423a-a876-d8869e5a11a8 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:07.983491 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 294/593] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:07.984539 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3145e7-58cf-423a-a876-d8869e5a11a8 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:08.038091 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee738c83-5589-4829-826a-4b397ea8babf None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:08.085149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82abeb-397d-422f-abec-ed7c97103460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:08.086201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82abeb-397d-422f-abec-ed7c97103460 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:08.086491 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82abeb-397d-422f-abec-ed7c97103460 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:08.087075 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82abeb-397d-422f-abec-ed7c97103460 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:08.092484 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:08.095825 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3145e7-58cf-423a-a876-d8869e5a11a8 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:08.103373 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82abeb-397d-422f-abec-ed7c97103460 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:08.105596 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db82abeb-397d-422f-abec-ed7c97103460 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:08.123549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-db82abeb-397d-422f-abec-ed7c97103460 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:08.123549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:08.123549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:08.123549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:08.123549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:08.123549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:08.123549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:08.123549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:08.123549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:08.123549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:08.123549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:08.123549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:08.123549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:08.123549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:08.123549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:08.124702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:08.124702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:08.124702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:08.124702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:08.124702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:08.124702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:08.124702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:08.124702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:08.124702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:08.124702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:08.124702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:08.124702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:08.124702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:08.124702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:08.124702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:08.124702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:08.124702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:08.125696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:08.125696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:08.125696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:08.125696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:08.125696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 300/594] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:08.126015 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82abeb-397d-422f-abec-ed7c97103460 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:08.147495 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ac52e-b1cc-43dd-9902-17e8619d59aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:08.148390 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ac52e-b1cc-43dd-9902-17e8619d59aa None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:08.148827 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ac52e-b1cc-43dd-9902-17e8619d59aa None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:08.149268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ac52e-b1cc-43dd-9902-17e8619d59aa None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:08.150793 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ac52e-b1cc-43dd-9902-17e8619d59aa None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:08.151731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ac52e-b1cc-43dd-9902-17e8619d59aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:08.152910 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:08.155286 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ac52e-b1cc-43dd-9902-17e8619d59aa None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:08.178016 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 295/595] 10.222.0.46 () {56 vars in 1139 bytes} [Tue Aug 2 08:47:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:08.179718 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ac52e-b1cc-43dd-9902-17e8619d59aa None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:08.234103 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db82abeb-397d-422f-abec-ed7c97103460 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:08.276369 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f93c19-755b-496a-9bcb-f89027fc0514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:08.276827 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f93c19-755b-496a-9bcb-f89027fc0514 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:08.277107 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f93c19-755b-496a-9bcb-f89027fc0514 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:08.277381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f93c19-755b-496a-9bcb-f89027fc0514 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:08.284198 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f93c19-755b-496a-9bcb-f89027fc0514 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:08.285645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f93c19-755b-496a-9bcb-f89027fc0514 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:08.297844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-41f93c19-755b-496a-9bcb-f89027fc0514 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:08.297844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:08.297844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:08.297844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:08.297844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:08.297844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:08.297844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:08.297844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:08.297844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:08.297844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:08.297844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:08.297844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:08.297844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:08.297844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:08.297844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:08.298980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:08.298980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:08.298980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:08.298980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:08.298980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:08.298980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:08.298980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:08.298980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:08.298980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:08.298980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:08.298980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:08.298980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:08.298980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:08.298980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:08.298980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:08.298980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:08.298980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:08.300016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:08.300016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:08.300016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:08.300016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:08.300016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:08.300016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:08.300016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:08.300016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:08.300016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 301/596] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:08.300547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f93c19-755b-496a-9bcb-f89027fc0514 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:08.305640 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ac52e-b1cc-43dd-9902-17e8619d59aa None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:08.344558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff70b6d-25fe-4dc2-898c-7ba92f4a051b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:08.345408 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff70b6d-25fe-4dc2-898c-7ba92f4a051b None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:08.345863 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff70b6d-25fe-4dc2-898c-7ba92f4a051b None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:08.346310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff70b6d-25fe-4dc2-898c-7ba92f4a051b None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:08.347250 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff70b6d-25fe-4dc2-898c-7ba92f4a051b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:08.347811 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff70b6d-25fe-4dc2-898c-7ba92f4a051b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:08.349320 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff70b6d-25fe-4dc2-898c-7ba92f4a051b None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:08.364031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 296/597] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:08.365211 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff70b6d-25fe-4dc2-898c-7ba92f4a051b None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:08.384134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:08.388420 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:08.402633 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f93c19-755b-496a-9bcb-f89027fc0514 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:08.436690 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca0cea-2b30-4af8-ad5d-2e614ec27007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:08.437436 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca0cea-2b30-4af8-ad5d-2e614ec27007 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:08.437615 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca0cea-2b30-4af8-ad5d-2e614ec27007 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:08.437876 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca0cea-2b30-4af8-ad5d-2e614ec27007 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:08.453722 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca0cea-2b30-4af8-ad5d-2e614ec27007 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:08.456223 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca0cea-2b30-4af8-ad5d-2e614ec27007 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:08.474700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-0bca0cea-2b30-4af8-ad5d-2e614ec27007 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:08.474700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:08.474700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:08.474700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:08.474700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:08.474700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:08.474700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:08.474700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:08.474700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:08.474700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:08.474700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:08.474700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:08.474700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:08.474700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:08.474700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:08.476256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:08.476256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:08.476256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:08.476256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:08.476256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:08.476256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:08.476256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:08.476256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:08.476256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:08.476256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:08.476256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:08.476256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:08.476256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:08.476256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:08.476256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:08.476256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:08.476256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:08.477152 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:08.477152 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:08.477152 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:08.477152 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:08.477152 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 302/598] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:08.477152 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca0cea-2b30-4af8-ad5d-2e614ec27007 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:08.482755 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff70b6d-25fe-4dc2-898c-7ba92f4a051b None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:08.518189 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf0815-00e4-414c-ae8e-3822e7c3f9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:08.518798 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf0815-00e4-414c-ae8e-3822e7c3f9ba None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:08.519073 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf0815-00e4-414c-ae8e-3822e7c3f9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:08.519343 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf0815-00e4-414c-ae8e-3822e7c3f9ba None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:08.520080 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf0815-00e4-414c-ae8e-3822e7c3f9ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:08.520387 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf0815-00e4-414c-ae8e-3822e7c3f9ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:08.522345 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf0815-00e4-414c-ae8e-3822e7c3f9ba None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:08.533931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 297/599] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:08.535105 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71fb1b-3a40-42c8-a339-dd2554046532 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:08.535385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71fb1b-3a40-42c8-a339-dd2554046532 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:08.535655 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71fb1b-3a40-42c8-a339-dd2554046532 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:08.539952 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 298/600] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:08.539952 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71fb1b-3a40-42c8-a339-dd2554046532 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:08.598453 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca0cea-2b30-4af8-ad5d-2e614ec27007 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:08.624684 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:08.638991 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66331c2-3a89-430a-b5ba-43e88a11bc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:08.639646 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66331c2-3a89-430a-b5ba-43e88a11bc71 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:08.639965 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66331c2-3a89-430a-b5ba-43e88a11bc71 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:08.640233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66331c2-3a89-430a-b5ba-43e88a11bc71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:08.642608 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:08.648698 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66331c2-3a89-430a-b5ba-43e88a11bc71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:08.649075 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71fb1b-3a40-42c8-a339-dd2554046532 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:08.650187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66331c2-3a89-430a-b5ba-43e88a11bc71 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:08.661960 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-e66331c2-3a89-430a-b5ba-43e88a11bc71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:08.661960 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:08.661960 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:08.661960 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:08.661960 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:08.661960 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:08.661960 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:08.661960 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:08.661960 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:08.661960 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:08.661960 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:08.661960 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:08.661960 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:08.661960 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:08.661960 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:08.663961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:08.663961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:08.663961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:08.663961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:08.663961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:08.663961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:08.663961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:08.663961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:08.663961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:08.663961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:08.663961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:08.663961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:08.663961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:08.663961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:08.663961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:08.663961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:08.663961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:08.664895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:08.664895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:08.664895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:08.664895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:08.664895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:08.664895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:08.664895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:08.664895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:08.664895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 303/601] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:08.665404 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66331c2-3a89-430a-b5ba-43e88a11bc71 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:08.694688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb49c6e4-fd63-4d69-92f8-91f0754f2d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:08.695309 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49c6e4-fd63-4d69-92f8-91f0754f2d54 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:08.695591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49c6e4-fd63-4d69-92f8-91f0754f2d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:08.695872 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49c6e4-fd63-4d69-92f8-91f0754f2d54 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:08.696563 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49c6e4-fd63-4d69-92f8-91f0754f2d54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:08.696870 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49c6e4-fd63-4d69-92f8-91f0754f2d54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:08.698524 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb49c6e4-fd63-4d69-92f8-91f0754f2d54 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:08.711288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 299/602] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:08.712978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4a2ce-9244-493b-8134-4655854adb37 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:08.713432 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4a2ce-9244-493b-8134-4655854adb37 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:08.713711 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4a2ce-9244-493b-8134-4655854adb37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:08.732770 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-1fd4a2ce-9244-493b-8134-4655854adb37 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:08.734007 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-1fd4a2ce-9244-493b-8134-4655854adb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:08.781170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66331c2-3a89-430a-b5ba-43e88a11bc71 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:08.813805 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 300/603] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:47:08.813805 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4a2ce-9244-493b-8134-4655854adb37 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:08.835845 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1646999-3ced-4af0-be32-bdd9f4b8e7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:08.836530 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1646999-3ced-4af0-be32-bdd9f4b8e7c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:08.836811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1646999-3ced-4af0-be32-bdd9f4b8e7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:08.837088 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1646999-3ced-4af0-be32-bdd9f4b8e7c6 None admin] PATH_INFO: `/v3/projects/4227a5b804c0414c99d407bb1415cf49/users/4abd1512ad224da896c6f2257f9602e3/roles/391becda2bf04438b8d1763ab99acf1b` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:08.853709 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1646999-3ced-4af0-be32-bdd9f4b8e7c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=4227a5b804c0414c99d407bb1415cf49, user_id=4abd1512ad224da896c6f2257f9602e3, role_id=391becda2bf04438b8d1763ab99acf1b)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:08.855094 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1646999-3ced-4af0-be32-bdd9f4b8e7c6 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:08.877052 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:08.909153 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4a2ce-9244-493b-8134-4655854adb37 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:08.930435 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 304/604] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:47:08 2022] PUT /identity/v3/projects/4227a5b804c0414c99d407bb1415cf49/users/4abd1512ad224da896c6f2257f9602e3/roles/391becda2bf04438b8d1763ab99acf1b => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:47:08.931279 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1646999-3ced-4af0-be32-bdd9f4b8e7c6 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:08.964556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e9ece-eef2-4a2a-a246-3654cc4e204f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:08.965703 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e9ece-eef2-4a2a-a246-3654cc4e204f None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:08.966559 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e9ece-eef2-4a2a-a246-3654cc4e204f None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:08.967345 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e9ece-eef2-4a2a-a246-3654cc4e204f None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:08.968302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e9ece-eef2-4a2a-a246-3654cc4e204f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:08.968942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e9ece-eef2-4a2a-a246-3654cc4e204f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:08.971344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123e9ece-eef2-4a2a-a246-3654cc4e204f None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:08.991369 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 301/605] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:08.992146 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:08.993720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e9ece-eef2-4a2a-a246-3654cc4e204f None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:09.052660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1646999-3ced-4af0-be32-bdd9f4b8e7c6 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:09.090944 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce1c92-0127-48dd-a9e9-6d9da6fdc01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:09.091611 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce1c92-0127-48dd-a9e9-6d9da6fdc01d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:09.091892 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce1c92-0127-48dd-a9e9-6d9da6fdc01d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:09.092163 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ce1c92-0127-48dd-a9e9-6d9da6fdc01d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:09.100477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce1c92-0127-48dd-a9e9-6d9da6fdc01d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:09.102039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ce1c92-0127-48dd-a9e9-6d9da6fdc01d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:09.113036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e9ece-eef2-4a2a-a246-3654cc4e204f None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:09.114454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-97ce1c92-0127-48dd-a9e9-6d9da6fdc01d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:09.114454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:09.114454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:09.114454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:09.114454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:09.114454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:09.114454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:09.114454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:09.114454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:09.114454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:09.114454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:09.114454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:09.114454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:09.114454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:09.114454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:09.115336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:09.115336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:09.115336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:09.115336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:09.115336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:09.115336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:09.115336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:09.115336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:09.115336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:09.115336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:09.115336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:09.115336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:09.115336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:09.115336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:09.115336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:09.115336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:09.115336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:09.117002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:09.117002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:09.117002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:09.117002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:09.117002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:09.117002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:09.117002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:09.117002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:09.120884 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 305/606] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:09.121818 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce1c92-0127-48dd-a9e9-6d9da6fdc01d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:09.151034 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:09.162042 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87488b-3d36-4aac-9210-9a7b2b003984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:09.163754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87488b-3d36-4aac-9210-9a7b2b003984 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:09.164108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87488b-3d36-4aac-9210-9a7b2b003984 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:09.164394 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce87488b-3d36-4aac-9210-9a7b2b003984 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:09.175690 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce87488b-3d36-4aac-9210-9a7b2b003984 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:47:09.181274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-ce87488b-3d36-4aac-9210-9a7b2b003984 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:09.181274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:09.181274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:09.181274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:09.181274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:09.181274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:09.181274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:09.181274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:09.181274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:09.181274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:09.181274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:09.181274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:09.181274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:47:09.181274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:47:09.181274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:47:09.181274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:47:09.182305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:47:09.182305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:47:09.182305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:47:09.182305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:47:09.182305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:09.182305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:09.182305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:09.182305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:09.182305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:09.182305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:09.182305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:09.182305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:09.182305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:09.182305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:09.182305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:09.182305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:09.183162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:09.183162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:09.183162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:09.183162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:09.183162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:47:09.183162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:47:09.183162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:47:09.183162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:47:09.183162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:47:09.183162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:47:09.183162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:09.183162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:09.184794 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 302/607] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 2 08:47:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:09.184794 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87488b-3d36-4aac-9210-9a7b2b003984 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:09.236343 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ce1c92-0127-48dd-a9e9-6d9da6fdc01d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:09.239752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:09.276048 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7c713-cde6-4aa1-ae89-8b14c427db2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:09.276723 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7c713-cde6-4aa1-ae89-8b14c427db2b None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:09.277165 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7c713-cde6-4aa1-ae89-8b14c427db2b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:09.277449 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7c713-cde6-4aa1-ae89-8b14c427db2b None admin] PATH_INFO: `/v3/projects/4227a5b804c0414c99d407bb1415cf49/users/6e39392b79bc439780415db7c8206098/roles/391becda2bf04438b8d1763ab99acf1b` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:09.296231 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7c713-cde6-4aa1-ae89-8b14c427db2b None admin] RBAC: Authorizing `identity:create_grant(project_id=4227a5b804c0414c99d407bb1415cf49, user_id=6e39392b79bc439780415db7c8206098, role_id=391becda2bf04438b8d1763ab99acf1b)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:09.296611 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce87488b-3d36-4aac-9210-9a7b2b003984 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:09.297685 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7c713-cde6-4aa1-ae89-8b14c427db2b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:09.331039 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d9044-27c3-464b-a54d-64a5415df075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:09.331658 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d9044-27c3-464b-a54d-64a5415df075 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:09.331944 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d9044-27c3-464b-a54d-64a5415df075 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:09.332219 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d9044-27c3-464b-a54d-64a5415df075 None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:09.332969 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d9044-27c3-464b-a54d-64a5415df075 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:09.333277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d9044-27c3-464b-a54d-64a5415df075 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:09.335297 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d9044-27c3-464b-a54d-64a5415df075 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:09.347265 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 303/608] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:09.348533 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c7b0e-d5c2-4e6c-9041-1bf46ec0838c None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:09.348827 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c7b0e-d5c2-4e6c-9041-1bf46ec0838c None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:09.350600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c7b0e-d5c2-4e6c-9041-1bf46ec0838c None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:09.355210 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 304/609] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:09.355210 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c7b0e-d5c2-4e6c-9041-1bf46ec0838c None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:09.404596 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:09.404887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 306/610] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:47:09 2022] PUT /identity/v3/projects/4227a5b804c0414c99d407bb1415cf49/users/6e39392b79bc439780415db7c8206098/roles/391becda2bf04438b8d1763ab99acf1b => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:47:09.414111 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7c713-cde6-4aa1-ae89-8b14c427db2b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:09.478578 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c7b0e-d5c2-4e6c-9041-1bf46ec0838c None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:09.529565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b86e4-807a-407f-81af-9f7768600bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:09.530502 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b86e4-807a-407f-81af-9f7768600bab None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:09.530783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b86e4-807a-407f-81af-9f7768600bab None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:09.531581 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b86e4-807a-407f-81af-9f7768600bab None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:09.532362 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b86e4-807a-407f-81af-9f7768600bab None admin] RBAC: Adding query filter params (name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:09.532943 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b86e4-807a-407f-81af-9f7768600bab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:09.534649 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7b86e4-807a-407f-81af-9f7768600bab None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:09.547910 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 305/611] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 2 08:47:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:09.549500 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc25e33-0cb4-4c85-a491-c23d720bcec9 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:09.549860 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc25e33-0cb4-4c85-a491-c23d720bcec9 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:09.550238 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc25e33-0cb4-4c85-a491-c23d720bcec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:09.557684 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7c713-cde6-4aa1-ae89-8b14c427db2b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:09.574503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-fcc25e33-0cb4-4c85-a491-c23d720bcec9 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:09.575871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-fcc25e33-0cb4-4c85-a491-c23d720bcec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:09.604521 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d29aaf-6510-4e80-8152-aba4773c65bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:09.605197 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d29aaf-6510-4e80-8152-aba4773c65bf None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:09.605485 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d29aaf-6510-4e80-8152-aba4773c65bf None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:09.605784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d29aaf-6510-4e80-8152-aba4773c65bf None admin] PATH_INFO: `/v3/projects/4227a5b804c0414c99d407bb1415cf49/users/ddf6f4d84bb34d2f810a0f6cec36da4d/roles/391becda2bf04438b8d1763ab99acf1b` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:09.627051 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d29aaf-6510-4e80-8152-aba4773c65bf None admin] RBAC: Authorizing `identity:create_grant(project_id=4227a5b804c0414c99d407bb1415cf49, user_id=ddf6f4d84bb34d2f810a0f6cec36da4d, role_id=391becda2bf04438b8d1763ab99acf1b)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:09.629657 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d29aaf-6510-4e80-8152-aba4773c65bf None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:09.663316 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 306/612] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:47:09.664991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc25e33-0cb4-4c85-a491-c23d720bcec9 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:09.719042 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:09.719812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 307/613] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:47:09 2022] PUT /identity/v3/projects/4227a5b804c0414c99d407bb1415cf49/users/ddf6f4d84bb34d2f810a0f6cec36da4d/roles/391becda2bf04438b8d1763ab99acf1b => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:47:09.722217 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d29aaf-6510-4e80-8152-aba4773c65bf None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:09.738234 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:09.854133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc25e33-0cb4-4c85-a491-c23d720bcec9 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:09.908410 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d29aaf-6510-4e80-8152-aba4773c65bf None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:09.911777 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf2261-9150-4a8f-ae4c-c9b336f502f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:09.913094 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf2261-9150-4a8f-ae4c-c9b336f502f6 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:09.913451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf2261-9150-4a8f-ae4c-c9b336f502f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:09.913676 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf2261-9150-4a8f-ae4c-c9b336f502f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:09.930135 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf2261-9150-4a8f-ae4c-c9b336f502f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:09.931696 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf2261-9150-4a8f-ae4c-c9b336f502f6 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:09.962687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-4edf2261-9150-4a8f-ae4c-c9b336f502f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:09.962687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:09.962687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:09.962687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:09.962687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:09.962687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:09.962687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:09.962687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:09.962687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:09.962687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:09.962687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:09.962687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:09.962687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:09.962687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:09.962687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:09.963668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:09.963668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:09.963668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:09.963668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:09.963668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:09.963668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:09.963668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:09.963668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:09.963668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:09.963668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:09.963668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:09.963668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:09.963668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:09.963668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:09.963668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:09.963668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:09.963668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:09.965158 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:09.965158 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:09.965158 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:09.965158 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:09.965158 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 307/614] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:09.969677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf2261-9150-4a8f-ae4c-c9b336f502f6 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:09.982669 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90016c1-d659-4ed2-aa9c-4b2a70baf0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:09.983637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90016c1-d659-4ed2-aa9c-4b2a70baf0cf None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:09.984129 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90016c1-d659-4ed2-aa9c-4b2a70baf0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:09.984561 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90016c1-d659-4ed2-aa9c-4b2a70baf0cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:10.007380 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90016c1-d659-4ed2-aa9c-4b2a70baf0cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:47:10.012549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-a90016c1-d659-4ed2-aa9c-4b2a70baf0cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:10.012549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:10.012549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:10.012549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:10.012549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:10.012549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:10.012549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:10.012549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:10.012549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:10.012549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:10.012549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:10.012549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:10.012549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:47:10.012549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:47:10.012549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:47:10.012549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:47:10.013480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:47:10.013480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:47:10.013480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:47:10.013480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:47:10.013480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:10.013480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:10.013480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:10.013480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:10.013480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:10.013480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:10.013480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:10.013480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:10.013480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:10.013480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:10.013480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:10.013480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:10.014364 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:10.014364 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:10.014364 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:10.014364 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:10.014364 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:47:10.014364 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:47:10.014364 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:47:10.014364 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:47:10.014364 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:47:10.014364 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:47:10.014364 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:10.014364 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:10.021079 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 308/615] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 2 08:47:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:10.022524 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90016c1-d659-4ed2-aa9c-4b2a70baf0cf None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:10.130786 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf2261-9150-4a8f-ae4c-c9b336f502f6 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:10.134299 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:10.134787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:10.182442 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26cda43-4342-49e4-baae-8885f36ffcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:10.182984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cda43-4342-49e4-baae-8885f36ffcfe None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:10.183360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cda43-4342-49e4-baae-8885f36ffcfe None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:10.183615 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cda43-4342-49e4-baae-8885f36ffcfe None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:10.184370 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26cda43-4342-49e4-baae-8885f36ffcfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:10.184676 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26cda43-4342-49e4-baae-8885f36ffcfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:10.186073 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26cda43-4342-49e4-baae-8885f36ffcfe None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:10.210609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90016c1-d659-4ed2-aa9c-4b2a70baf0cf None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:10.212300 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 308/616] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:10.214042 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26cda43-4342-49e4-baae-8885f36ffcfe None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:10.283162 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bee03d-61fd-4e44-b166-05799b01e79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:10.284072 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bee03d-61fd-4e44-b166-05799b01e79d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:10.284507 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bee03d-61fd-4e44-b166-05799b01e79d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:10.284953 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bee03d-61fd-4e44-b166-05799b01e79d None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:10.285901 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bee03d-61fd-4e44-b166-05799b01e79d None admin] RBAC: Adding query filter params (name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:10.286359 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bee03d-61fd-4e44-b166-05799b01e79d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:10.287988 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bee03d-61fd-4e44-b166-05799b01e79d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:10.309985 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 309/617] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 2 08:47:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:10.314257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bee03d-61fd-4e44-b166-05799b01e79d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:10.414151 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26cda43-4342-49e4-baae-8885f36ffcfe None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:10.439491 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bee03d-61fd-4e44-b166-05799b01e79d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:10.474657 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:10.485712 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60cb919-23a3-47dd-a845-bb3442e5471c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:10.495282 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cb919-23a3-47dd-a845-bb3442e5471c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:10.495282 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cb919-23a3-47dd-a845-bb3442e5471c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:10.495282 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cb919-23a3-47dd-a845-bb3442e5471c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:10.501390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60cb919-23a3-47dd-a845-bb3442e5471c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:10.503418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60cb919-23a3-47dd-a845-bb3442e5471c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:10.511412 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa9ccb-7623-49b5-8ae5-188a9044bbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:10.512397 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa9ccb-7623-49b5-8ae5-188a9044bbeb None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:10.512890 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa9ccb-7623-49b5-8ae5-188a9044bbeb None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:10.513370 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa9ccb-7623-49b5-8ae5-188a9044bbeb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:10.519356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-c60cb919-23a3-47dd-a845-bb3442e5471c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:10.519356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:10.519356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:10.519356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:10.519356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:10.519356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:10.519356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:10.519356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:10.519356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:10.519356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:10.519356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:10.519356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:10.519356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:10.519356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:10.519356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:10.520418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:10.520418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:10.520418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:10.520418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:10.520418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:10.520418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:10.520418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:10.520418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:10.520418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:10.520418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:10.520418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:10.520418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:10.520418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:10.520418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:10.520418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:10.520418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:10.520418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:10.521780 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:10.521780 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:10.521780 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:10.521780 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:10.530126 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 310/618] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:10.533429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa9ccb-7623-49b5-8ae5-188a9044bbeb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:10.535667 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60cb919-23a3-47dd-a845-bb3442e5471c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:10.539715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfa9ccb-7623-49b5-8ae5-188a9044bbeb None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:10.561487 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-6bfa9ccb-7623-49b5-8ae5-188a9044bbeb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:47:10.561487 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:10.561487 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:10.561487 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:10.561487 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:10.561487 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:10.561487 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:10.561487 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:10.561487 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:10.561487 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:10.561487 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:10.561487 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:10.561487 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:10.561487 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:10.561487 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:10.561487 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:10.562532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:10.562532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:10.562532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:10.562532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:10.562532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:10.562532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:10.562532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:10.562532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:10.562532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:10.562532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:10.562532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:10.562532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:10.562532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:10.562532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:10.562532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:10.562532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:10.562532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:10.562532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:10.563909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:10.563909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:10.563909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:10.563909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:10.563909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:10.563909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:10.563909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:10.563909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:10.563909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:47:10.563909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:10.563909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 309/619] 10.222.0.46 () {56 vars in 1093 bytes} [Tue Aug 2 08:47:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:10.568125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa9ccb-7623-49b5-8ae5-188a9044bbeb None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:10.595683 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:10.708471 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60cb919-23a3-47dd-a845-bb3442e5471c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:10.738520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa9ccb-7623-49b5-8ae5-188a9044bbeb None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:10.783797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ad192-462a-4527-b1e1-2ed4f3cf36c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:10.784671 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ad192-462a-4527-b1e1-2ed4f3cf36c3 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:10.785106 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ad192-462a-4527-b1e1-2ed4f3cf36c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:10.785751 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ad192-462a-4527-b1e1-2ed4f3cf36c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:10.786784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ad192-462a-4527-b1e1-2ed4f3cf36c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:10.788812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ad192-462a-4527-b1e1-2ed4f3cf36c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:10.790609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ad192-462a-4527-b1e1-2ed4f3cf36c3 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:10.812771 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 311/620] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:10.814829 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f8672-c1f4-4853-80c7-1533338afba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:10.815618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f8672-c1f4-4853-80c7-1533338afba1 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:10.816103 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f8672-c1f4-4853-80c7-1533338afba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:10.816527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933f8672-c1f4-4853-80c7-1533338afba1 None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:10.818203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ad192-462a-4527-b1e1-2ed4f3cf36c3 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:10.819442 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f8672-c1f4-4853-80c7-1533338afba1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:10.821838 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f8672-c1f4-4853-80c7-1533338afba1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:10.823723 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933f8672-c1f4-4853-80c7-1533338afba1 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:10.854763 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 310/621] 10.222.0.46 () {56 vars in 1133 bytes} [Tue Aug 2 08:47:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:10.862401 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f8672-c1f4-4853-80c7-1533338afba1 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:10.886233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:10.956120 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:10.988397 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933f8672-c1f4-4853-80c7-1533338afba1 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:11.029057 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ad192-462a-4527-b1e1-2ed4f3cf36c3 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:11.044605 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dca8dd-67a0-4219-b145-62f79930a123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:11.045228 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dca8dd-67a0-4219-b145-62f79930a123 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:11.045504 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dca8dd-67a0-4219-b145-62f79930a123 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:11.045781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dca8dd-67a0-4219-b145-62f79930a123 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:11.064849 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a6fbc-c5eb-4a48-ad56-cf9bd8230fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:11.065768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a6fbc-c5eb-4a48-ad56-cf9bd8230fee None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:11.066227 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a6fbc-c5eb-4a48-ad56-cf9bd8230fee None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:11.066654 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a6fbc-c5eb-4a48-ad56-cf9bd8230fee None admin] PATH_INFO: `/v3/users/swift` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:11.075426 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a6fbc-c5eb-4a48-ad56-cf9bd8230fee None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:11.077064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926a6fbc-c5eb-4a48-ad56-cf9bd8230fee None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:11.090204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-926a6fbc-c5eb-4a48-ad56-cf9bd8230fee None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 08:47:11.090204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:11.090204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:11.090204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:11.090204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:11.090204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:11.090204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:11.090204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:11.090204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:11.090204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:11.090204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:11.090204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:11.090204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:11.090204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:11.090204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:11.090204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:11.091386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:11.091386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:11.091386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:11.091386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:11.091386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:11.091386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:11.091386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:11.091386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:11.091386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:11.091386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:11.091386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:11.091386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:11.091386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:11.091386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:11.091386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:11.091386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:11.091386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:11.091386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:11.092481 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:11.092481 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:11.092481 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:11.092481 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:11.092481 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:11.092481 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:11.092481 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:11.092481 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:11.092481 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 08:47:11.092481 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:11.094692 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 312/622] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:47:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:11.099868 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a6fbc-c5eb-4a48-ad56-cf9bd8230fee None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:11.173428 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dca8dd-67a0-4219-b145-62f79930a123 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:11.175190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dca8dd-67a0-4219-b145-62f79930a123 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:11.242450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:11.246438 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a6fbc-c5eb-4a48-ad56-cf9bd8230fee None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:11.251889 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:11.282318 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4bc9a3-1cf9-4401-a222-fe295cd6700c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:11.283097 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bc9a3-1cf9-4401-a222-fe295cd6700c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:11.283488 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bc9a3-1cf9-4401-a222-fe295cd6700c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:11.283950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bc9a3-1cf9-4401-a222-fe295cd6700c None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:11.284901 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4bc9a3-1cf9-4401-a222-fe295cd6700c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:11.285254 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4bc9a3-1cf9-4401-a222-fe295cd6700c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:11.286839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4bc9a3-1cf9-4401-a222-fe295cd6700c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:11.288296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 311/623] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:47:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:47:11.289842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd8ea8-1344-4458-96fe-09dd4ce136f5 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:11.290303 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd8ea8-1344-4458-96fe-09dd4ce136f5 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:11.290719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd8ea8-1344-4458-96fe-09dd4ce136f5 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:11.292544 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 312/624] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:11.293502 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd8ea8-1344-4458-96fe-09dd4ce136f5 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:11.306540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 313/625] 10.222.0.46 () {56 vars in 1135 bytes} [Tue Aug 2 08:47:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:11.307951 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa461a9-ffc8-49ff-8a7b-8ec22f35bdc2 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:11.308343 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa461a9-ffc8-49ff-8a7b-8ec22f35bdc2 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:11.308722 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa461a9-ffc8-49ff-8a7b-8ec22f35bdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:11.327812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5fa461a9-ffc8-49ff-8a7b-8ec22f35bdc2 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:11.329303 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5fa461a9-ffc8-49ff-8a7b-8ec22f35bdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:11.399901 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 314/626] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:47:11.400895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa461a9-ffc8-49ff-8a7b-8ec22f35bdc2 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:11.406121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd8ea8-1344-4458-96fe-09dd4ce136f5 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:11.442927 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4c52a-3e19-4c5f-b693-f87c8192a522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:11.443881 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4c52a-3e19-4c5f-b693-f87c8192a522 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:11.444316 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4c52a-3e19-4c5f-b693-f87c8192a522 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:11.444745 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4c52a-3e19-4c5f-b693-f87c8192a522 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:11.455912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f4c52a-3e19-4c5f-b693-f87c8192a522 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:11.457578 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f4c52a-3e19-4c5f-b693-f87c8192a522 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:11.473409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-e9f4c52a-3e19-4c5f-b693-f87c8192a522 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:11.473409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:11.473409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:11.473409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:11.473409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:11.473409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:11.473409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:11.473409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:11.473409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:11.473409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:11.473409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:11.473409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:11.473409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:11.473409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:11.473409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:11.474373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:11.474373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:11.474373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:11.474373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:11.474373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:11.474373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:11.474373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:11.474373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:11.474373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:11.474373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:11.474373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:11.474373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:11.474373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:11.474373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:11.474373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:11.474373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:11.474373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:11.475349 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:11.475349 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:11.475349 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:11.475349 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:11.477515 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 313/627] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:11.478680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4c52a-3e19-4c5f-b693-f87c8192a522 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:11.502373 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:11.504738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:11.507742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa461a9-ffc8-49ff-8a7b-8ec22f35bdc2 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:11.544857 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dac6e8-b75a-4aa5-8309-6775528d01c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:11.545546 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dac6e8-b75a-4aa5-8309-6775528d01c6 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:11.545832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dac6e8-b75a-4aa5-8309-6775528d01c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:11.546159 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dac6e8-b75a-4aa5-8309-6775528d01c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:11.557642 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dac6e8-b75a-4aa5-8309-6775528d01c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:11.559369 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dac6e8-b75a-4aa5-8309-6775528d01c6 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:11.575915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-e4dac6e8-b75a-4aa5-8309-6775528d01c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:11.575915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:11.575915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:11.575915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:11.575915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:11.575915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:11.575915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:11.575915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:11.575915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:11.575915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:11.575915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:11.575915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:11.575915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:11.575915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:11.575915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:11.576958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:11.576958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:11.576958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:11.576958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:11.576958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:11.576958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:11.576958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:11.576958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:11.576958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:11.576958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:11.576958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:11.576958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:11.576958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:11.576958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:11.576958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:11.576958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:11.576958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:11.577875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:11.577875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:11.577875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:11.577875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:11.577875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 315/628] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:11.577875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dac6e8-b75a-4aa5-8309-6775528d01c6 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:11.610213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4c52a-3e19-4c5f-b693-f87c8192a522 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:11.655300 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afc55a-ca7b-431c-99b2-7403e2db7297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:11.656411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afc55a-ca7b-431c-99b2-7403e2db7297 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:11.657640 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afc55a-ca7b-431c-99b2-7403e2db7297 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:11.658136 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afc55a-ca7b-431c-99b2-7403e2db7297 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:11.671085 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afc55a-ca7b-431c-99b2-7403e2db7297 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:47:11.677402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-e2afc55a-ca7b-431c-99b2-7403e2db7297 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:11.677402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:11.677402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:11.677402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:11.677402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:11.677402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:11.677402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:11.677402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:11.677402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:11.677402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:11.677402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:11.677402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:11.677402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:47:11.677402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:47:11.677402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:47:11.677402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:47:11.679036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:47:11.679036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:47:11.679036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:47:11.679036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:47:11.679036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:11.679036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:11.679036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:11.679036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:11.679036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:11.679036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:11.679036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:11.679036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:11.679036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:11.679036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:11.679036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:11.679036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:11.679906 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:11.679906 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:11.679906 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:11.679906 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:11.679906 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:47:11.679906 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:47:11.679906 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:47:11.679906 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:47:11.679906 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:47:11.679906 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:47:11.679906 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:11.679906 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:11.679906 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 314/629] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 2 08:47:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:11.680629 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afc55a-ca7b-431c-99b2-7403e2db7297 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:11.693026 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dac6e8-b75a-4aa5-8309-6775528d01c6 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:11.728451 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd6d2e-6296-403e-9f32-648e807e43ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:11.729114 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd6d2e-6296-403e-9f32-648e807e43ef None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:11.729198 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd6d2e-6296-403e-9f32-648e807e43ef None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:11.729430 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd6d2e-6296-403e-9f32-648e807e43ef None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:11.730142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd6d2e-6296-403e-9f32-648e807e43ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:11.730455 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd6d2e-6296-403e-9f32-648e807e43ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:11.731850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd6d2e-6296-403e-9f32-648e807e43ef None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:11.739441 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:11.744990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 316/630] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:11.746294 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae8d66-bdfe-47c1-9058-b1ad7f7c8a6b None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:11.746512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae8d66-bdfe-47c1-9058-b1ad7f7c8a6b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:11.746793 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae8d66-bdfe-47c1-9058-b1ad7f7c8a6b None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:11.748288 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 317/631] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:11.749143 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae8d66-bdfe-47c1-9058-b1ad7f7c8a6b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:11.783792 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:11.802088 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afc55a-ca7b-431c-99b2-7403e2db7297 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:11.840186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9200ff-5943-4f23-8ac4-488054e08e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:11.840802 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9200ff-5943-4f23-8ac4-488054e08e2d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:11.841090 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9200ff-5943-4f23-8ac4-488054e08e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:11.841370 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9200ff-5943-4f23-8ac4-488054e08e2d None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:11.842094 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9200ff-5943-4f23-8ac4-488054e08e2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:11.842408 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9200ff-5943-4f23-8ac4-488054e08e2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:11.843771 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9200ff-5943-4f23-8ac4-488054e08e2d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:11.860419 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 315/632] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:11.860419 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9200ff-5943-4f23-8ac4-488054e08e2d None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:11.865733 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae8d66-bdfe-47c1-9058-b1ad7f7c8a6b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:11.903876 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e348f-2e82-40fd-b9bb-c0263a4ccb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:11.904501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e348f-2e82-40fd-b9bb-c0263a4ccb5b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:11.904784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e348f-2e82-40fd-b9bb-c0263a4ccb5b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:11.905113 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e348f-2e82-40fd-b9bb-c0263a4ccb5b None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:11.905837 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e348f-2e82-40fd-b9bb-c0263a4ccb5b None admin] RBAC: Adding query filter params (name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:11.906212 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e348f-2e82-40fd-b9bb-c0263a4ccb5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:11.907584 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e348f-2e82-40fd-b9bb-c0263a4ccb5b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:11.920708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 318/633] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 2 08:47:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:11.921731 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b64504-0caf-48c2-8f9a-021a8786438b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:11.922047 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b64504-0caf-48c2-8f9a-021a8786438b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:11.922327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b64504-0caf-48c2-8f9a-021a8786438b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:11.941710 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-54b64504-0caf-48c2-8f9a-021a8786438b None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:11.943011 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-54b64504-0caf-48c2-8f9a-021a8786438b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:11.968549 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9200ff-5943-4f23-8ac4-488054e08e2d None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:12.001143 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:12.006721 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948e899-45c0-41fb-89a6-394946bb171d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:12.013052 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948e899-45c0-41fb-89a6-394946bb171d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:12.013052 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948e899-45c0-41fb-89a6-394946bb171d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:12.013052 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948e899-45c0-41fb-89a6-394946bb171d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:12.022602 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 319/634] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:47:12.022602 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b64504-0caf-48c2-8f9a-021a8786438b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:12.024586 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948e899-45c0-41fb-89a6-394946bb171d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:12.026555 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948e899-45c0-41fb-89a6-394946bb171d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:12.040542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-d948e899-45c0-41fb-89a6-394946bb171d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:12.040542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:12.040542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:12.040542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:12.040542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:12.040542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:12.040542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:12.040542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:12.040542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:12.040542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:12.040542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:12.040542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:12.040542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:12.040542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:12.040542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:12.041536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:12.041536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:12.041536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:12.041536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:12.041536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:12.041536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:12.041536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:12.041536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:12.041536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:12.041536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:12.041536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:12.041536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:12.041536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:12.041536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:12.041536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:12.041536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:12.041536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:12.042499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:12.042499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:12.042499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:12.042499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:12.042499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:12.042499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:12.042499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:12.042499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:12.046431 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 316/635] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:12.047270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948e899-45c0-41fb-89a6-394946bb171d None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:12.050603 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:12.131385 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b64504-0caf-48c2-8f9a-021a8786438b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:12.154414 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948e899-45c0-41fb-89a6-394946bb171d None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:12.165625 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d10fa-1443-42bf-85b7-71386e6476ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:12.166293 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d10fa-1443-42bf-85b7-71386e6476ba None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:12.166572 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d10fa-1443-42bf-85b7-71386e6476ba None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:12.166843 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d10fa-1443-42bf-85b7-71386e6476ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:12.173464 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d10fa-1443-42bf-85b7-71386e6476ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:12.174961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d10fa-1443-42bf-85b7-71386e6476ba None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:12.186954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-c02d10fa-1443-42bf-85b7-71386e6476ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:12.186954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:12.186954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:12.186954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:12.186954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:12.186954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:12.186954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:12.186954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:12.186954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:12.186954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:12.186954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:12.186954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:12.186954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:12.186954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:12.186954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:12.188063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:12.188063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:12.188063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:12.188063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:12.188063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:12.188063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:12.188063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:12.188063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:12.188063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:12.188063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:12.188063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:12.188063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:12.188063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:12.188063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:12.188063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:12.188063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:12.188063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:12.188986 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:12.188986 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:12.188986 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:12.188986 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:12.188986 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:12.188986 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:12.188986 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:12.188986 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:12.188986 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 320/636] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:12.188986 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d10fa-1443-42bf-85b7-71386e6476ba None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:12.190221 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b927ccd9-8dbd-455f-8ab4-2193b8857d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:12.190862 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927ccd9-8dbd-455f-8ab4-2193b8857d7b None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:12.191148 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927ccd9-8dbd-455f-8ab4-2193b8857d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:12.191427 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b927ccd9-8dbd-455f-8ab4-2193b8857d7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:12.202549 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b927ccd9-8dbd-455f-8ab4-2193b8857d7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:12.203990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b927ccd9-8dbd-455f-8ab4-2193b8857d7b None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:12.218717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-b927ccd9-8dbd-455f-8ab4-2193b8857d7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:12.218717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:12.218717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:12.218717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:12.218717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:12.218717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:12.218717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:12.218717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:12.218717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:12.218717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:12.218717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:12.218717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:12.218717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:12.218717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:12.218717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:12.219670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:12.219670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:12.219670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:12.219670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:12.219670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:12.219670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:12.219670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:12.219670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:12.219670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:12.219670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:12.219670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:12.219670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:12.219670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:12.219670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:12.219670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:12.219670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:12.219670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:12.220570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:12.220570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:12.220570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:12.220570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:12.220570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 317/637] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:12.220860 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b927ccd9-8dbd-455f-8ab4-2193b8857d7b None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:12.243019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:12.276390 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:12.290916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d10fa-1443-42bf-85b7-71386e6476ba None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:12.320650 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b927ccd9-8dbd-455f-8ab4-2193b8857d7b None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:12.325236 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5437e68-68e5-41b7-8fb4-d80cdad1d147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:12.325876 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5437e68-68e5-41b7-8fb4-d80cdad1d147 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:12.326183 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5437e68-68e5-41b7-8fb4-d80cdad1d147 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:12.326457 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5437e68-68e5-41b7-8fb4-d80cdad1d147 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:12.337019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5437e68-68e5-41b7-8fb4-d80cdad1d147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:47:12.341875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-f5437e68-68e5-41b7-8fb4-d80cdad1d147 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:12.341875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:12.341875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:12.341875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:12.341875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:12.341875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:12.341875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:12.341875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:12.341875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:12.341875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:12.341875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:12.341875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:12.341875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:47:12.341875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:47:12.341875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:47:12.341875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:47:12.343008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:47:12.343008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:47:12.343008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:47:12.343008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:47:12.343008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:12.343008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:12.343008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:12.343008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:12.343008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:12.343008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:12.343008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:12.343008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:12.343008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:12.343008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:12.343008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:12.343008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:12.343886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:12.343886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:12.343886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:12.343886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:12.343886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:47:12.343886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:47:12.343886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:47:12.343886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:47:12.343886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:47:12.343886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:47:12.343886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:12.343886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:12.343886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 321/638] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 2 08:47:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:12.344602 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfc06e-22fc-4eca-bcc1-dd5ff183a9ee None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:12.344878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfc06e-22fc-4eca-bcc1-dd5ff183a9ee None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:12.345208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfc06e-22fc-4eca-bcc1-dd5ff183a9ee None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:12.346505 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 322/639] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:12.347080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbfc06e-22fc-4eca-bcc1-dd5ff183a9ee None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:12.360945 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea168d3-bbde-444c-988c-e18484179171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:12.361565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea168d3-bbde-444c-988c-e18484179171 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:12.361847 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea168d3-bbde-444c-988c-e18484179171 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:12.362148 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea168d3-bbde-444c-988c-e18484179171 None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:12.362892 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea168d3-bbde-444c-988c-e18484179171 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:12.363200 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea168d3-bbde-444c-988c-e18484179171 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:12.364622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea168d3-bbde-444c-988c-e18484179171 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:12.376373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 318/640] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:12.377155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea168d3-bbde-444c-988c-e18484179171 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:12.450796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbfc06e-22fc-4eca-bcc1-dd5ff183a9ee None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:12.462243 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:12.477822 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea168d3-bbde-444c-988c-e18484179171 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:12.485175 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3caf86-af3c-4282-b281-f4fbb3eb8fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:12.485787 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3caf86-af3c-4282-b281-f4fbb3eb8fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:12.486108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3caf86-af3c-4282-b281-f4fbb3eb8fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:12.486385 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3caf86-af3c-4282-b281-f4fbb3eb8fe9 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:12.487135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3caf86-af3c-4282-b281-f4fbb3eb8fe9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:12.487442 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3caf86-af3c-4282-b281-f4fbb3eb8fe9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:12.488884 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3caf86-af3c-4282-b281-f4fbb3eb8fe9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:12.491560 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:12.500854 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 323/641] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:12.501637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3caf86-af3c-4282-b281-f4fbb3eb8fe9 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:12.516537 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b225711-ef94-4deb-8334-f1aa65f006af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:12.517170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b225711-ef94-4deb-8334-f1aa65f006af None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:12.517657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b225711-ef94-4deb-8334-f1aa65f006af None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:12.518198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b225711-ef94-4deb-8334-f1aa65f006af None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:12.519206 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b225711-ef94-4deb-8334-f1aa65f006af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:12.519550 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b225711-ef94-4deb-8334-f1aa65f006af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:12.520908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b225711-ef94-4deb-8334-f1aa65f006af None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:12.532493 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 319/642] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:12.533642 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92d4ba-bfd4-4060-9671-8ec14901cd0e None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:12.533952 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92d4ba-bfd4-4060-9671-8ec14901cd0e None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:12.534396 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92d4ba-bfd4-4060-9671-8ec14901cd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:12.551453 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-cd92d4ba-bfd4-4060-9671-8ec14901cd0e None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:12.552564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-cd92d4ba-bfd4-4060-9671-8ec14901cd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:12.602554 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3caf86-af3c-4282-b281-f4fbb3eb8fe9 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:12.624489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 320/643] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:47:12.625333 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92d4ba-bfd4-4060-9671-8ec14901cd0e None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:12.637061 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a6141-e5ce-43b8-b914-97b87590d452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:12.637683 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a6141-e5ce-43b8-b914-97b87590d452 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:12.638021 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a6141-e5ce-43b8-b914-97b87590d452 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:12.638275 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a6141-e5ce-43b8-b914-97b87590d452 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:12.639012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a6141-e5ce-43b8-b914-97b87590d452 None admin] RBAC: Adding query filter params (name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:12.639304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a6141-e5ce-43b8-b914-97b87590d452 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:12.640670 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a6141-e5ce-43b8-b914-97b87590d452 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:12.649602 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 324/644] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 2 08:47:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:12.650417 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a6141-e5ce-43b8-b914-97b87590d452 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:12.680391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:12.734900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92d4ba-bfd4-4060-9671-8ec14901cd0e None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:12.737153 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:12.749262 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a6141-e5ce-43b8-b914-97b87590d452 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:12.769296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7048a-eec1-4f15-b554-7cea0943ce7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:12.769919 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7048a-eec1-4f15-b554-7cea0943ce7a None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:12.770203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7048a-eec1-4f15-b554-7cea0943ce7a None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:12.770474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7048a-eec1-4f15-b554-7cea0943ce7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:12.771249 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7048a-eec1-4f15-b554-7cea0943ce7a None admin] RBAC: Adding query filter params (role.id=391becda2bf04438b8d1763ab99acf1b, scope.project.id=4227a5b804c0414c99d407bb1415cf49, user.id=3191d804e8dc4b1a85f490319833ec07) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:12.771553 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7048a-eec1-4f15-b554-7cea0943ce7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:12.773604 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7048a-eec1-4f15-b554-7cea0943ce7a None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:12.779177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 321/645] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:47:12 2022] GET /identity/v3/role_assignments?user.id=3191d804e8dc4b1a85f490319833ec07&scope.project.id=4227a5b804c0414c99d407bb1415cf49&role.id=391becda2bf04438b8d1763ab99acf1b => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:12.779936 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7048a-eec1-4f15-b554-7cea0943ce7a None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:12.784140 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb918b-0d63-426c-a36c-af8b86468ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:12.784838 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb918b-0d63-426c-a36c-af8b86468ceb None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:12.785125 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb918b-0d63-426c-a36c-af8b86468ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:12.785407 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb918b-0d63-426c-a36c-af8b86468ceb None admin] PATH_INFO: `/v3/projects/4227a5b804c0414c99d407bb1415cf49/users/ff92ef3b9da74ccab3fbc43f852e3ad0/roles/391becda2bf04438b8d1763ab99acf1b` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:12.802511 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb918b-0d63-426c-a36c-af8b86468ceb None admin] RBAC: Authorizing `identity:create_grant(project_id=4227a5b804c0414c99d407bb1415cf49, user_id=ff92ef3b9da74ccab3fbc43f852e3ad0, role_id=391becda2bf04438b8d1763ab99acf1b)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:12.803946 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddb918b-0d63-426c-a36c-af8b86468ceb None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:12.861661 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 325/646] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:47:12 2022] PUT /identity/v3/projects/4227a5b804c0414c99d407bb1415cf49/users/ff92ef3b9da74ccab3fbc43f852e3ad0/roles/391becda2bf04438b8d1763ab99acf1b => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:47:12.862946 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb918b-0d63-426c-a36c-af8b86468ceb None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:13.006827 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:13.028742 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7048a-eec1-4f15-b554-7cea0943ce7a None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:13.033317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb918b-0d63-426c-a36c-af8b86468ceb None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:13.079704 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d5dc1-672c-4000-9f2d-2c07186fcce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:13.080325 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d5dc1-672c-4000-9f2d-2c07186fcce7 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:13.080559 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d5dc1-672c-4000-9f2d-2c07186fcce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:13.080918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d5dc1-672c-4000-9f2d-2c07186fcce7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:13.082864 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79ebdd-2dbd-46cf-bed5-3fcf42863464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:13.083578 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79ebdd-2dbd-46cf-bed5-3fcf42863464 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:13.083925 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79ebdd-2dbd-46cf-bed5-3fcf42863464 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:13.084851 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79ebdd-2dbd-46cf-bed5-3fcf42863464 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:13.089858 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d5dc1-672c-4000-9f2d-2c07186fcce7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:13.090467 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d5dc1-672c-4000-9f2d-2c07186fcce7 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:13.096417 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79ebdd-2dbd-46cf-bed5-3fcf42863464 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:47:13.101833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-3e79ebdd-2dbd-46cf-bed5-3fcf42863464 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:13.101833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:13.101833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:13.101833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:13.101833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:13.101833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:13.101833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:13.101833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:13.101833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:13.101833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:13.101833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:13.101833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:13.101833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:47:13.101833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:47:13.101833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:47:13.101833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:47:13.102802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:47:13.102802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:47:13.102802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:47:13.102802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:47:13.102802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:13.102802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:13.102802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:13.102802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:13.102802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:13.102802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:13.102802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:13.102802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:13.102802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:13.102802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:13.102802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:13.102802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:13.103694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:13.103694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:13.103694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:13.103694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:13.103694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:47:13.103694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:47:13.103694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:47:13.103694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:47:13.103694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:47:13.103694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:47:13.103694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:13.103694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:13.104356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-d03d5dc1-672c-4000-9f2d-2c07186fcce7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 02 08:47:13.104356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:13.104356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:13.104356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:13.104356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:13.104356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:13.104356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:13.104356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:13.104356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:13.104356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:13.104356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:13.104356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:13.104356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:13.104356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:13.104356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:13.104356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:13.105193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:13.105193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:13.105193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:13.105193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:13.105193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:13.105193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:13.105193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:13.105193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:13.105193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:13.105193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:13.105193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:13.105193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:13.105193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:13.105193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:13.105193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:13.105193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:13.105193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:13.105193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:13.106177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:13.106177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:13.106177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:13.106177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:13.106177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:13.106177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:13.106177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:13.106177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:13.106177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 02 08:47:13.106177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:13.106721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 326/647] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 2 08:47:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:13.106721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79ebdd-2dbd-46cf-bed5-3fcf42863464 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:13.106845 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 322/648] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:47:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:13.109701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d5dc1-672c-4000-9f2d-2c07186fcce7 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:13.146501 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:13.224711 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79ebdd-2dbd-46cf-bed5-3fcf42863464 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:13.225453 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d5dc1-672c-4000-9f2d-2c07186fcce7 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:13.261301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0966ab4-139c-4aaa-8d1f-7e701471621e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:13.262207 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ad53a-b839-449c-87e6-b04009dcd39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:13.262432 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0966ab4-139c-4aaa-8d1f-7e701471621e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:13.262852 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0966ab4-139c-4aaa-8d1f-7e701471621e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:13.263112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ad53a-b839-449c-87e6-b04009dcd39d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:13.263419 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0966ab4-139c-4aaa-8d1f-7e701471621e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:13.263545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ad53a-b839-449c-87e6-b04009dcd39d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:13.263947 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ad53a-b839-449c-87e6-b04009dcd39d None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:13.264863 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ad53a-b839-449c-87e6-b04009dcd39d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:13.265271 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ad53a-b839-449c-87e6-b04009dcd39d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:13.266828 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ad53a-b839-449c-87e6-b04009dcd39d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:13.276050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0966ab4-139c-4aaa-8d1f-7e701471621e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:13.277719 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0966ab4-139c-4aaa-8d1f-7e701471621e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:13.279720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:13.286683 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 323/649] 10.222.0.46 () {56 vars in 1137 bytes} [Tue Aug 2 08:47:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:13.287631 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ad53a-b839-449c-87e6-b04009dcd39d None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:13.294215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-b0966ab4-139c-4aaa-8d1f-7e701471621e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:13.294215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:13.294215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:13.294215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:13.294215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:13.294215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:13.294215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:13.294215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:13.294215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:13.294215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:13.294215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:13.294215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:13.294215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:13.294215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:13.294215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:13.309082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:13.309082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:13.309082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:13.309082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:13.309082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:13.309082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:13.309082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:13.309082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:13.309082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:13.309082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:13.309082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:13.309082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:13.309082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:13.309082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:13.309082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:13.309082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:13.309082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:13.310017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:13.310017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:13.310017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:13.310017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:13.310017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 327/650] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:13.310017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0966ab4-139c-4aaa-8d1f-7e701471621e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:13.382492 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:13.398240 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ad53a-b839-449c-87e6-b04009dcd39d None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:13.401568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0966ab4-139c-4aaa-8d1f-7e701471621e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:13.434395 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e0647-c365-4aba-afbf-11967aec7fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:13.435371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e0647-c365-4aba-afbf-11967aec7fdb None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:13.435843 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e0647-c365-4aba-afbf-11967aec7fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:13.436417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e0647-c365-4aba-afbf-11967aec7fdb None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:13.437148 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e0647-c365-4aba-afbf-11967aec7fdb None admin] RBAC: Adding query filter params (name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:13.437461 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e0647-c365-4aba-afbf-11967aec7fdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:13.437713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c466dcb-6889-4a1a-9e30-a2c9fe55ed81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:13.438431 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c466dcb-6889-4a1a-9e30-a2c9fe55ed81 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:13.438775 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c466dcb-6889-4a1a-9e30-a2c9fe55ed81 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:13.438927 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e0647-c365-4aba-afbf-11967aec7fdb None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:13.439185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c466dcb-6889-4a1a-9e30-a2c9fe55ed81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:13.449990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 324/651] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 2 08:47:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:13.450840 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e0647-c365-4aba-afbf-11967aec7fdb None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:13.506735 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:13.514986 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c466dcb-6889-4a1a-9e30-a2c9fe55ed81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:13.516659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c466dcb-6889-4a1a-9e30-a2c9fe55ed81 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:13.579773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e0647-c365-4aba-afbf-11967aec7fdb None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:13.620006 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6f38b-b07b-43e4-b612-7116b12007e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:13.620900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6f38b-b07b-43e4-b612-7116b12007e5 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:13.621431 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6f38b-b07b-43e4-b612-7116b12007e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:13.621853 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb6f38b-b07b-43e4-b612-7116b12007e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:13.622878 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6f38b-b07b-43e4-b612-7116b12007e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:13.623418 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6f38b-b07b-43e4-b612-7116b12007e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:13.625082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb6f38b-b07b-43e4-b612-7116b12007e5 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:13.638680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 325/652] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:13.640020 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6f38b-b07b-43e4-b612-7116b12007e5 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:13.645757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 328/653] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:47:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:47:13.647331 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c466dcb-6889-4a1a-9e30-a2c9fe55ed81 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:13.656416 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:13.761440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:13.762138 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb6f38b-b07b-43e4-b612-7116b12007e5 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:13.770263 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c466dcb-6889-4a1a-9e30-a2c9fe55ed81 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:13.805124 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279eb5a-b1d5-4f1e-86d7-f586e67ede6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:13.805774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279eb5a-b1d5-4f1e-86d7-f586e67ede6c None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:13.806089 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279eb5a-b1d5-4f1e-86d7-f586e67ede6c None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:13.806371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279eb5a-b1d5-4f1e-86d7-f586e67ede6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:13.813056 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a5047-7526-4097-8c80-e4f9b5d06a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:13.813953 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a5047-7526-4097-8c80-e4f9b5d06a63 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:13.815008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a5047-7526-4097-8c80-e4f9b5d06a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:13.815587 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31a5047-7526-4097-8c80-e4f9b5d06a63 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:13.818247 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279eb5a-b1d5-4f1e-86d7-f586e67ede6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:13.820628 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b279eb5a-b1d5-4f1e-86d7-f586e67ede6c None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:13.825307 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a5047-7526-4097-8c80-e4f9b5d06a63 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:13.826979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31a5047-7526-4097-8c80-e4f9b5d06a63 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:13.837411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-b279eb5a-b1d5-4f1e-86d7-f586e67ede6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:13.837411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:13.837411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:13.837411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:13.837411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:13.837411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:13.837411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:13.837411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:13.837411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:13.837411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:13.837411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:13.837411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:13.837411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:13.837411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:13.837411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:13.838357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:13.838357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:13.838357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:13.838357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:13.838357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:13.838357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:13.838357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:13.838357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:13.838357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:13.838357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:13.838357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:13.838357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:13.838357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:13.838357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:13.838357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:13.838357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:13.838357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:13.839255 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:13.839255 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:13.839255 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:13.839255 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:13.839255 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 326/654] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:13.844634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279eb5a-b1d5-4f1e-86d7-f586e67ede6c None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:13.844773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-e31a5047-7526-4097-8c80-e4f9b5d06a63 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 08:47:13.844773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:13.844773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:13.844773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:13.844773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:13.844773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:13.844773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:13.844773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:13.844773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:13.844773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:13.844773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:13.844773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:13.844773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:13.844773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:13.844773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:13.844773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:13.845634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:13.845634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:13.845634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:13.845634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:13.845634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:13.845634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:13.845634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:13.845634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:13.845634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:13.845634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:13.845634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:13.845634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:13.845634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:13.845634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:13.845634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:13.845634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:13.845634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:13.845634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:13.847800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:13.847800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:13.847800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:13.847800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:13.847800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:13.847800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:13.847800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:13.847800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:13.847800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 08:47:13.847800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:13.847800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 329/655] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:47:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:13.847800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a5047-7526-4097-8c80-e4f9b5d06a63 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:13.942385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:13.956346 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31a5047-7526-4097-8c80-e4f9b5d06a63 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:13.974382 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279eb5a-b1d5-4f1e-86d7-f586e67ede6c None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:13.992534 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8c73c-341d-4302-9e14-69f5695b5cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:13.993055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8c73c-341d-4302-9e14-69f5695b5cbf None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:13.993339 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8c73c-341d-4302-9e14-69f5695b5cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:13.993659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e8c73c-341d-4302-9e14-69f5695b5cbf None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:13.994473 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8c73c-341d-4302-9e14-69f5695b5cbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:13.994744 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8c73c-341d-4302-9e14-69f5695b5cbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:13.996118 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e8c73c-341d-4302-9e14-69f5695b5cbf None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:14.013265 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 330/656] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:14.013265 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8c73c-341d-4302-9e14-69f5695b5cbf None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:14.017158 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b2f11-d76a-42fa-b187-ba7af71aa43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:14.018097 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b2f11-d76a-42fa-b187-ba7af71aa43f None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:14.018696 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b2f11-d76a-42fa-b187-ba7af71aa43f None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:14.019175 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b2f11-d76a-42fa-b187-ba7af71aa43f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:14.026476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:14.032147 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b2f11-d76a-42fa-b187-ba7af71aa43f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:14.034190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b2f11-d76a-42fa-b187-ba7af71aa43f None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:14.049622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-916b2f11-d76a-42fa-b187-ba7af71aa43f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:14.049622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:14.049622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:14.049622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:14.049622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:14.049622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:14.049622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:14.049622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:14.049622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:14.049622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:14.049622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:14.049622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:14.049622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:14.049622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:14.049622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:14.050701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:14.050701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:14.050701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:14.050701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:14.050701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:14.050701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:14.050701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:14.050701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:14.050701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:14.050701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:14.050701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:14.050701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:14.050701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:14.050701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:14.050701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:14.050701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:14.050701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:14.051670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:14.051670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:14.051670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:14.051670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:14.054362 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 327/657] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:14.055631 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b2f11-d76a-42fa-b187-ba7af71aa43f None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:14.121232 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e8c73c-341d-4302-9e14-69f5695b5cbf None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:14.157153 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c663620-3331-4cae-b78d-47999b7f6651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:14.157852 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c663620-3331-4cae-b78d-47999b7f6651 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:14.158081 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c663620-3331-4cae-b78d-47999b7f6651 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:14.158362 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c663620-3331-4cae-b78d-47999b7f6651 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:14.159156 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c663620-3331-4cae-b78d-47999b7f6651 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:14.159463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c663620-3331-4cae-b78d-47999b7f6651 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:14.160949 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b2f11-d76a-42fa-b187-ba7af71aa43f None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:14.161069 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c663620-3331-4cae-b78d-47999b7f6651 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:14.179356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 331/658] 10.222.0.46 () {56 vars in 1137 bytes} [Tue Aug 2 08:47:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:14.180358 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c663620-3331-4cae-b78d-47999b7f6651 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:14.195228 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:14.202528 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495d725-b61c-460e-8d97-ceb9d5526c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:14.204650 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495d725-b61c-460e-8d97-ceb9d5526c39 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:14.205150 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495d725-b61c-460e-8d97-ceb9d5526c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:14.205920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0495d725-b61c-460e-8d97-ceb9d5526c39 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:14.216782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0495d725-b61c-460e-8d97-ceb9d5526c39 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:14.219709 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0495d725-b61c-460e-8d97-ceb9d5526c39 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:14.235587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-0495d725-b61c-460e-8d97-ceb9d5526c39 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 08:47:14.235587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:14.235587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:14.235587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:14.235587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:14.235587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:14.235587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:14.235587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:14.235587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:14.235587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:14.235587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:14.235587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:14.235587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:14.235587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:14.235587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:14.235587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:14.236552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:14.236552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:14.236552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:14.236552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:14.236552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:14.236552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:14.236552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:14.236552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:14.236552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:14.236552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:14.236552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:14.236552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:14.236552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:14.236552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:14.236552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:14.236552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:14.236552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:14.236552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:14.237499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:14.237499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:14.237499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:14.237499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:14.237499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:14.237499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:14.237499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:14.237499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:14.237499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 08:47:14.237499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:14.244459 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 328/659] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 2 08:47:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:14.244459 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495d725-b61c-460e-8d97-ceb9d5526c39 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:14.268014 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:14.309345 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c663620-3331-4cae-b78d-47999b7f6651 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:14.354970 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0495d725-b61c-460e-8d97-ceb9d5526c39 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:14.356773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff505e8-e7ab-4653-8020-d8087e9b3aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:14.358454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff505e8-e7ab-4653-8020-d8087e9b3aff None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:14.358900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff505e8-e7ab-4653-8020-d8087e9b3aff None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:14.359326 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff505e8-e7ab-4653-8020-d8087e9b3aff None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:14.360168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff505e8-e7ab-4653-8020-d8087e9b3aff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:14.360626 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff505e8-e7ab-4653-8020-d8087e9b3aff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:14.362215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff505e8-e7ab-4653-8020-d8087e9b3aff None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:14.379016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 332/660] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:14.380206 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff505e8-e7ab-4653-8020-d8087e9b3aff None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:14.396742 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5fc2b-05aa-40d8-9c43-d79d263c9482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:14.397535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5fc2b-05aa-40d8-9c43-d79d263c9482 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:14.397951 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5fc2b-05aa-40d8-9c43-d79d263c9482 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:14.398384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f5fc2b-05aa-40d8-9c43-d79d263c9482 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:14.446966 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:14.474213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f5fc2b-05aa-40d8-9c43-d79d263c9482 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:14.475784 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f5fc2b-05aa-40d8-9c43-d79d263c9482 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:14.511424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff505e8-e7ab-4653-8020-d8087e9b3aff None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:14.547180 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:14.552889 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289ff2d-8a07-489f-97dc-92d3092e8a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:14.553821 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289ff2d-8a07-489f-97dc-92d3092e8a63 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:14.554228 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289ff2d-8a07-489f-97dc-92d3092e8a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:14.554850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b289ff2d-8a07-489f-97dc-92d3092e8a63 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:14.555755 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b289ff2d-8a07-489f-97dc-92d3092e8a63 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:14.556266 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b289ff2d-8a07-489f-97dc-92d3092e8a63 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:14.557987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b289ff2d-8a07-489f-97dc-92d3092e8a63 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:14.582069 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 333/661] 10.222.0.46 () {56 vars in 1139 bytes} [Tue Aug 2 08:47:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:14.583100 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289ff2d-8a07-489f-97dc-92d3092e8a63 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:14.609044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 329/662] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:47:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:47:14.610782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5fc2b-05aa-40d8-9c43-d79d263c9482 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:14.688889 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:14.709401 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b289ff2d-8a07-489f-97dc-92d3092e8a63 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:14.725456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f5fc2b-05aa-40d8-9c43-d79d263c9482 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:14.749146 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6a496-9068-4b9c-bc02-b478c1c07342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:14.749973 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6a496-9068-4b9c-bc02-b478c1c07342 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:14.750357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6a496-9068-4b9c-bc02-b478c1c07342 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:14.750739 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff6a496-9068-4b9c-bc02-b478c1c07342 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:14.758435 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6a496-9068-4b9c-bc02-b478c1c07342 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:14.760108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff6a496-9068-4b9c-bc02-b478c1c07342 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:14.767746 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cdc280-f273-448a-b56a-f560767fdc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:14.768555 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdc280-f273-448a-b56a-f560767fdc98 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:14.768993 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdc280-f273-448a-b56a-f560767fdc98 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:14.769431 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdc280-f273-448a-b56a-f560767fdc98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:14.774019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-dff6a496-9068-4b9c-bc02-b478c1c07342 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:14.774019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:14.774019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:14.774019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:14.774019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:14.774019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:14.774019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:14.774019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:14.774019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:14.774019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:14.774019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:14.774019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:14.774019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:14.774019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:14.774019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:14.775434 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:14.775434 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:14.775434 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:14.775434 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:14.775434 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:14.775434 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:14.775434 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:14.775434 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:14.775434 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:14.775434 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:14.775434 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:14.775434 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:14.775434 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:14.775434 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:14.775434 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:14.775434 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:14.775434 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:14.776528 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:14.776528 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:14.776528 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:14.776528 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:14.776528 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:14.776528 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:14.776528 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:14.776528 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:14.782304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 334/663] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:14.783244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6a496-9068-4b9c-bc02-b478c1c07342 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:14.839912 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:14.861984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cdc280-f273-448a-b56a-f560767fdc98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:14.864543 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cdc280-f273-448a-b56a-f560767fdc98 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:14.895789 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff6a496-9068-4b9c-bc02-b478c1c07342 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:14.945992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd1e9d-78b6-4112-a2b3-8a055b91b164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:14.947269 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd1e9d-78b6-4112-a2b3-8a055b91b164 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:14.947771 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd1e9d-78b6-4112-a2b3-8a055b91b164 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:14.948519 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd1e9d-78b6-4112-a2b3-8a055b91b164 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:14.963035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd1e9d-78b6-4112-a2b3-8a055b91b164 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:14.964399 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:14.965127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bd1e9d-78b6-4112-a2b3-8a055b91b164 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:14.981808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-39bd1e9d-78b6-4112-a2b3-8a055b91b164 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:14.981808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:14.981808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:14.981808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:14.981808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:14.981808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:14.981808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:14.981808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:14.981808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:14.981808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:14.981808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:14.981808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:14.981808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:14.981808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:14.981808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:14.982796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:14.982796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:14.982796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:14.982796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:14.982796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:14.982796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:14.982796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:14.982796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:14.982796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:14.982796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:14.982796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:14.982796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:14.982796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:14.982796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:14.982796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:14.982796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:14.982796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:14.983878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:14.983878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:14.983878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:14.983878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:14.983878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 335/664] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:14.984190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd1e9d-78b6-4112-a2b3-8a055b91b164 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:15.005051 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 330/665] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:47:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:47:15.006956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e318c56-b050-496c-b0c2-e22a5ae4bce1 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:15.007245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e318c56-b050-496c-b0c2-e22a5ae4bce1 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:15.007661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e318c56-b050-496c-b0c2-e22a5ae4bce1 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:15.009153 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 331/666] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:15.009812 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e318c56-b050-496c-b0c2-e22a5ae4bce1 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:15.091661 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bd1e9d-78b6-4112-a2b3-8a055b91b164 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:15.093135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:15.113211 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e318c56-b050-496c-b0c2-e22a5ae4bce1 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:15.125347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20115eef-9f7c-4da2-ae7f-e925c7351380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:15.125989 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20115eef-9f7c-4da2-ae7f-e925c7351380 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:15.126277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20115eef-9f7c-4da2-ae7f-e925c7351380 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:15.126558 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20115eef-9f7c-4da2-ae7f-e925c7351380 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:15.127310 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20115eef-9f7c-4da2-ae7f-e925c7351380 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:15.127620 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20115eef-9f7c-4da2-ae7f-e925c7351380 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:15.129044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20115eef-9f7c-4da2-ae7f-e925c7351380 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:15.141153 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 336/667] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:15.142034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20115eef-9f7c-4da2-ae7f-e925c7351380 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:15.148142 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e550ae1-e209-421d-a23a-0be5751c1f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:15.148729 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e550ae1-e209-421d-a23a-0be5751c1f0c None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:15.149008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e550ae1-e209-421d-a23a-0be5751c1f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:15.149283 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e550ae1-e209-421d-a23a-0be5751c1f0c None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:15.150006 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e550ae1-e209-421d-a23a-0be5751c1f0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:15.150312 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e550ae1-e209-421d-a23a-0be5751c1f0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:15.151775 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e550ae1-e209-421d-a23a-0be5751c1f0c None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:15.163849 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 332/668] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:15.165069 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ecec9-3fb2-4a62-8634-7b2cdff59bfd None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:15.165354 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ecec9-3fb2-4a62-8634-7b2cdff59bfd None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:15.165623 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ecec9-3fb2-4a62-8634-7b2cdff59bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:15.183001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-343ecec9-3fb2-4a62-8634-7b2cdff59bfd None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:15.184182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-343ecec9-3fb2-4a62-8634-7b2cdff59bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:15.210520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:15.245196 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20115eef-9f7c-4da2-ae7f-e925c7351380 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:15.250990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 333/669] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:47:15.252368 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4fbeb-8dd1-45c0-b520-db5e87313617 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:15.252651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4fbeb-8dd1-45c0-b520-db5e87313617 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:15.252936 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4fbeb-8dd1-45c0-b520-db5e87313617 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:15.254492 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 334/670] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:15.255172 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4fbeb-8dd1-45c0-b520-db5e87313617 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:15.283748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0925c1c-134b-44ba-9f8f-6689041d0a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:15.284390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0925c1c-134b-44ba-9f8f-6689041d0a31 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:15.284676 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0925c1c-134b-44ba-9f8f-6689041d0a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:15.284956 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0925c1c-134b-44ba-9f8f-6689041d0a31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:15.295406 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0925c1c-134b-44ba-9f8f-6689041d0a31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:15.296866 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0925c1c-134b-44ba-9f8f-6689041d0a31 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:15.311229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-c0925c1c-134b-44ba-9f8f-6689041d0a31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:15.311229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:15.311229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:15.311229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:15.311229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:15.311229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:15.311229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:15.311229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:15.311229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:15.311229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:15.311229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:15.311229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:15.311229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:15.311229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:15.311229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:15.312295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:15.312295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:15.312295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:15.312295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:15.312295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:15.312295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:15.312295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:15.312295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:15.312295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:15.312295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:15.312295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:15.312295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:15.312295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:15.312295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:15.312295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:15.312295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:15.312295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:15.313360 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:15.313360 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:15.313360 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:15.313360 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:15.313360 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 337/671] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:15.313360 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0925c1c-134b-44ba-9f8f-6689041d0a31 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:15.328917 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:15.369381 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4fbeb-8dd1-45c0-b520-db5e87313617 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:15.403373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b7611-1057-4bb9-9695-2364639ef96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:15.403977 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b7611-1057-4bb9-9695-2364639ef96c None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:15.404254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b7611-1057-4bb9-9695-2364639ef96c None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:15.404527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b7611-1057-4bb9-9695-2364639ef96c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:15.405304 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b7611-1057-4bb9-9695-2364639ef96c None admin] RBAC: Adding query filter params (role.id=391becda2bf04438b8d1763ab99acf1b, scope.project.id=4227a5b804c0414c99d407bb1415cf49, user.id=4abd1512ad224da896c6f2257f9602e3) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:15.405613 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b7611-1057-4bb9-9695-2364639ef96c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:15.407076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59b7611-1057-4bb9-9695-2364639ef96c None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:15.413414 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0925c1c-134b-44ba-9f8f-6689041d0a31 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:15.429918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 335/672] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:47:15 2022] GET /identity/v3/role_assignments?user.id=4abd1512ad224da896c6f2257f9602e3&scope.project.id=4227a5b804c0414c99d407bb1415cf49&role.id=391becda2bf04438b8d1763ab99acf1b => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:15.429918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8509383-8742-4e08-814f-e9b4c020c38a None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:15.429918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8509383-8742-4e08-814f-e9b4c020c38a None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:15.429918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8509383-8742-4e08-814f-e9b4c020c38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:15.433921 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-a8509383-8742-4e08-814f-e9b4c020c38a None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:15.435157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-a8509383-8742-4e08-814f-e9b4c020c38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:15.453973 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e2358-635d-43dd-9a18-ce43e1c42f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:15.454633 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e2358-635d-43dd-9a18-ce43e1c42f2f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:15.454915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e2358-635d-43dd-9a18-ce43e1c42f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:15.455191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e2358-635d-43dd-9a18-ce43e1c42f2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:15.458865 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:15.463217 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e2358-635d-43dd-9a18-ce43e1c42f2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:15.464736 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e2358-635d-43dd-9a18-ce43e1c42f2f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:15.478304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-507e2358-635d-43dd-9a18-ce43e1c42f2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:15.478304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:15.478304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:15.478304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:15.478304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:15.478304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:15.478304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:15.478304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:15.478304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:15.478304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:15.478304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:15.478304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:15.478304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:15.478304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:15.478304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:15.485012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:15.485012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:15.485012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:15.485012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:15.485012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:15.485012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:15.485012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:15.485012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:15.485012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:15.485012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:15.485012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:15.485012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:15.485012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:15.485012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:15.485012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:15.485012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:15.485012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:15.486289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:15.486289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:15.486289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:15.486289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:15.486289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:15.486289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:15.486289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:15.486289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:15.486289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 338/673] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:15.486289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e2358-635d-43dd-9a18-ce43e1c42f2f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:15.516159 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 336/674] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:47:15.517286 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8509383-8742-4e08-814f-e9b4c020c38a None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:15.586709 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:15.613590 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e2358-635d-43dd-9a18-ce43e1c42f2f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:15.637469 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8509383-8742-4e08-814f-e9b4c020c38a None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:15.653963 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d4e2b-328e-48f9-9d11-c3441577b95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:15.654611 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d4e2b-328e-48f9-9d11-c3441577b95d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:15.654894 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d4e2b-328e-48f9-9d11-c3441577b95d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:15.655157 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d4e2b-328e-48f9-9d11-c3441577b95d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:15.668117 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d4e2b-328e-48f9-9d11-c3441577b95d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:47:15.672987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-323d4e2b-328e-48f9-9d11-c3441577b95d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:15.672987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:15.672987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:15.672987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:15.672987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:15.672987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:15.672987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:15.672987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:15.672987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:15.672987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:15.672987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:15.672987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:15.672987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:47:15.672987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:47:15.672987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:47:15.672987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:47:15.673999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:47:15.673999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:47:15.673999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:47:15.673999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:47:15.673999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:15.673999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:15.673999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:15.673999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:15.673999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:15.673999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:15.673999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:15.673999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:15.673999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:15.673999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:15.673999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:15.673999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:15.674862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:15.674862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:15.674862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:15.674862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:15.674862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:47:15.674862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:47:15.674862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:47:15.674862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:47:15.674862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:47:15.674862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:47:15.674862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:15.674862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:15.674862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 339/675] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 2 08:47:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:15.675675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d4e2b-328e-48f9-9d11-c3441577b95d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:15.683371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9827ea50-52b1-4ad7-9b27-9ea1fd3670ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:15.684209 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827ea50-52b1-4ad7-9b27-9ea1fd3670ae None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:15.684329 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827ea50-52b1-4ad7-9b27-9ea1fd3670ae None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:15.684584 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827ea50-52b1-4ad7-9b27-9ea1fd3670ae None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:15.685329 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9827ea50-52b1-4ad7-9b27-9ea1fd3670ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:15.685634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9827ea50-52b1-4ad7-9b27-9ea1fd3670ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:15.687089 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9827ea50-52b1-4ad7-9b27-9ea1fd3670ae None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:15.699986 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 337/676] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:15.704941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9827ea50-52b1-4ad7-9b27-9ea1fd3670ae None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:15.725268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:15.781345 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d4e2b-328e-48f9-9d11-c3441577b95d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:15.810624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9827ea50-52b1-4ad7-9b27-9ea1fd3670ae None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:15.816595 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35252e-69b3-4ad1-b010-d0d66b28e85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:15.817222 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35252e-69b3-4ad1-b010-d0d66b28e85b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:15.817508 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35252e-69b3-4ad1-b010-d0d66b28e85b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:15.817788 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b35252e-69b3-4ad1-b010-d0d66b28e85b None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:15.818627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b35252e-69b3-4ad1-b010-d0d66b28e85b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:15.818937 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b35252e-69b3-4ad1-b010-d0d66b28e85b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:15.820391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b35252e-69b3-4ad1-b010-d0d66b28e85b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:15.822076 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:15.835829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 340/677] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:15.836672 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35252e-69b3-4ad1-b010-d0d66b28e85b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:15.853114 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df250181-9d60-4d9a-a53e-fd519bb32052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:15.853840 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df250181-9d60-4d9a-a53e-fd519bb32052 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:15.854173 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df250181-9d60-4d9a-a53e-fd519bb32052 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:15.854541 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df250181-9d60-4d9a-a53e-fd519bb32052 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:15.866351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df250181-9d60-4d9a-a53e-fd519bb32052 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:47:15.871182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-df250181-9d60-4d9a-a53e-fd519bb32052 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:47:15.871182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:15.871182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:15.871182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:15.871182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:15.871182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:15.871182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:15.871182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:15.871182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:15.871182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:15.871182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:15.871182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:15.871182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:47:15.871182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:47:15.871182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:47:15.871182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:47:15.872125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:47:15.872125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:47:15.872125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:47:15.872125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:47:15.872125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:15.872125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:15.872125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:15.872125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:15.872125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:15.872125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:15.872125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:15.872125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:15.872125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:15.872125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:15.872125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:15.872125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:15.872984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:15.872984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:15.872984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:15.872984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:15.872984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:47:15.872984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:47:15.872984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:47:15.872984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:47:15.872984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:47:15.872984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:47:15.872984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:47:15.872984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:15.877201 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 338/678] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:47:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:15.878162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df250181-9d60-4d9a-a53e-fd519bb32052 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:15.954213 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b35252e-69b3-4ad1-b010-d0d66b28e85b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:15.968178 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:15.989669 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8bf759-f7f0-4cad-a9d7-c73d95d73011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:15.990816 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bf759-f7f0-4cad-a9d7-c73d95d73011 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:15.991111 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bf759-f7f0-4cad-a9d7-c73d95d73011 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:15.991387 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bf759-f7f0-4cad-a9d7-c73d95d73011 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:15.992380 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8bf759-f7f0-4cad-a9d7-c73d95d73011 None admin] RBAC: Adding query filter params (name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:15.992683 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8bf759-f7f0-4cad-a9d7-c73d95d73011 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:15.994075 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8bf759-f7f0-4cad-a9d7-c73d95d73011 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:15.995287 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df250181-9d60-4d9a-a53e-fd519bb32052 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:16.002548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 341/679] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 2 08:47:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:16.003620 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8bf759-f7f0-4cad-a9d7-c73d95d73011 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:16.038206 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb2f7a-ed35-4c60-b218-a6f898e91bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:16.038798 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb2f7a-ed35-4c60-b218-a6f898e91bfb None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:16.039077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb2f7a-ed35-4c60-b218-a6f898e91bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:16.039353 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cb2f7a-ed35-4c60-b218-a6f898e91bfb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:16.046057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb2f7a-ed35-4c60-b218-a6f898e91bfb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:16.047473 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cb2f7a-ed35-4c60-b218-a6f898e91bfb None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:16.058535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-34cb2f7a-ed35-4c60-b218-a6f898e91bfb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:16.058535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:16.058535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:16.058535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:16.058535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:16.058535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:16.058535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:16.058535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:16.058535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:16.058535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:16.058535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:16.058535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:16.058535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:16.058535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:16.058535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:16.059626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:16.059626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:16.059626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:16.059626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:16.059626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:16.059626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:16.059626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:16.059626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:16.059626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:16.059626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:16.059626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:16.059626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:16.059626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:16.059626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:16.059626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:16.059626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:16.059626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:16.060754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:16.060754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:16.060754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:16.060754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:16.060754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:16.060754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:16.060754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:16.060754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:16.064320 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 339/680] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:16.065132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb2f7a-ed35-4c60-b218-a6f898e91bfb None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:16.069339 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:16.117908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8bf759-f7f0-4cad-a9d7-c73d95d73011 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:16.152474 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591bb0c-24eb-4325-8677-91b6ccdc6fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:16.153088 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591bb0c-24eb-4325-8677-91b6ccdc6fea None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:16.153364 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591bb0c-24eb-4325-8677-91b6ccdc6fea None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:16.153640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e591bb0c-24eb-4325-8677-91b6ccdc6fea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:16.154444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e591bb0c-24eb-4325-8677-91b6ccdc6fea None admin] RBAC: Adding query filter params (role.id=391becda2bf04438b8d1763ab99acf1b, scope.project.id=4227a5b804c0414c99d407bb1415cf49, user.id=6e39392b79bc439780415db7c8206098) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:16.154750 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e591bb0c-24eb-4325-8677-91b6ccdc6fea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:16.156177 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e591bb0c-24eb-4325-8677-91b6ccdc6fea None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:16.161829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 342/681] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:47:16 2022] GET /identity/v3/role_assignments?user.id=6e39392b79bc439780415db7c8206098&scope.project.id=4227a5b804c0414c99d407bb1415cf49&role.id=391becda2bf04438b8d1763ab99acf1b => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:16.163231 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591bb0c-24eb-4325-8677-91b6ccdc6fea None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:16.182547 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cb2f7a-ed35-4c60-b218-a6f898e91bfb None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:16.221630 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c9350-f184-4522-9bdb-5e2c77bb7e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:16.222295 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c9350-f184-4522-9bdb-5e2c77bb7e16 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:16.222581 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c9350-f184-4522-9bdb-5e2c77bb7e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:16.222862 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c9350-f184-4522-9bdb-5e2c77bb7e16 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:16.223603 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c9350-f184-4522-9bdb-5e2c77bb7e16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:16.223915 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c9350-f184-4522-9bdb-5e2c77bb7e16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:16.225952 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c9350-f184-4522-9bdb-5e2c77bb7e16 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:16.229000 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:16.237045 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 340/682] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:47:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:16.237733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c9350-f184-4522-9bdb-5e2c77bb7e16 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:16.327034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e591bb0c-24eb-4325-8677-91b6ccdc6fea None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:16.352732 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:16.356668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c9350-f184-4522-9bdb-5e2c77bb7e16 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:16.370385 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991cb730-f01c-4a1e-af7b-bc5b81085702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:16.371123 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991cb730-f01c-4a1e-af7b-bc5b81085702 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:16.371480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991cb730-f01c-4a1e-af7b-bc5b81085702 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:16.371826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991cb730-f01c-4a1e-af7b-bc5b81085702 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:16.385752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991cb730-f01c-4a1e-af7b-bc5b81085702 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:16.387763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991cb730-f01c-4a1e-af7b-bc5b81085702 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:16.399927 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744d806-35e9-4012-beae-c71f640561b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:16.400776 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744d806-35e9-4012-beae-c71f640561b2 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:16.401209 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744d806-35e9-4012-beae-c71f640561b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:16.401632 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744d806-35e9-4012-beae-c71f640561b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:16.402558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744d806-35e9-4012-beae-c71f640561b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:16.403031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744d806-35e9-4012-beae-c71f640561b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:16.404170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-991cb730-f01c-4a1e-af7b-bc5b81085702 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:16.404170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:16.404170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:16.404170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:16.404170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:16.404170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:16.404170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:16.404170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:16.404170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:16.404170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:16.404170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:16.404170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:16.404170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:16.404170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:16.404170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:16.405210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:16.405210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:16.405210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:16.405210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:16.405210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:16.405210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:16.405210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:16.405210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:16.405210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:16.405210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:16.405210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:16.405210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:16.405210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:16.405210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:16.405210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:16.405210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:16.405210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:16.406164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:16.406164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:16.406164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:16.406164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:16.406164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 343/683] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:16.406164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991cb730-f01c-4a1e-af7b-bc5b81085702 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:16.407022 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744d806-35e9-4012-beae-c71f640561b2 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:16.424180 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 341/684] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:16.428719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744d806-35e9-4012-beae-c71f640561b2 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:16.501720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:16.535978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991cb730-f01c-4a1e-af7b-bc5b81085702 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:16.555269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744d806-35e9-4012-beae-c71f640561b2 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:16.583291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a79d54-fd2c-4280-bb5a-8b077352625e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:16.584390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a79d54-fd2c-4280-bb5a-8b077352625e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:16.585211 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a79d54-fd2c-4280-bb5a-8b077352625e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:16.585779 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a79d54-fd2c-4280-bb5a-8b077352625e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:16.597232 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061ced3-23ab-45be-a672-4ed33e534e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:16.598002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061ced3-23ab-45be-a672-4ed33e534e50 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:16.598361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061ced3-23ab-45be-a672-4ed33e534e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:16.598713 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1061ced3-23ab-45be-a672-4ed33e534e50 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:16.600256 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061ced3-23ab-45be-a672-4ed33e534e50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:16.600407 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061ced3-23ab-45be-a672-4ed33e534e50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:16.601923 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1061ced3-23ab-45be-a672-4ed33e534e50 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:16.602801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a79d54-fd2c-4280-bb5a-8b077352625e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:16.605481 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a79d54-fd2c-4280-bb5a-8b077352625e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:16.617947 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 342/685] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:16.618954 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061ced3-23ab-45be-a672-4ed33e534e50 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:16.625933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-b8a79d54-fd2c-4280-bb5a-8b077352625e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:16.625933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:16.625933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:16.625933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:16.625933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:16.625933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:16.625933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:16.625933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:16.625933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:16.625933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:16.625933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:16.625933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:16.625933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:16.625933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:16.625933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:16.626992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:16.626992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:16.626992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:16.626992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:16.626992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:16.626992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:16.626992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:16.626992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:16.626992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:16.626992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:16.626992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:16.626992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:16.626992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:16.626992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:16.626992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:16.626992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:16.626992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:16.628097 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:16.628097 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:16.628097 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:16.628097 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:16.629865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 344/686] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:16.630915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a79d54-fd2c-4280-bb5a-8b077352625e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:16.674305 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:16.746477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a79d54-fd2c-4280-bb5a-8b077352625e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:16.761248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1061ced3-23ab-45be-a672-4ed33e534e50 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:16.782828 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045b85e-61d2-4440-9d4f-e347a0a0c662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:16.783619 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045b85e-61d2-4440-9d4f-e347a0a0c662 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:16.784002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045b85e-61d2-4440-9d4f-e347a0a0c662 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:16.784375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9045b85e-61d2-4440-9d4f-e347a0a0c662 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:16.784717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:16.794698 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045b85e-61d2-4440-9d4f-e347a0a0c662 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:16.797027 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9045b85e-61d2-4440-9d4f-e347a0a0c662 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:16.805379 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555fe137-82d0-46cb-9918-c231f25fed6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:16.806252 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fe137-82d0-46cb-9918-c231f25fed6c None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:16.806677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fe137-82d0-46cb-9918-c231f25fed6c None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:16.807092 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fe137-82d0-46cb-9918-c231f25fed6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:16.808012 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555fe137-82d0-46cb-9918-c231f25fed6c None admin] RBAC: Adding query filter params (role.id=391becda2bf04438b8d1763ab99acf1b, scope.project.id=4227a5b804c0414c99d407bb1415cf49, user.id=ddf6f4d84bb34d2f810a0f6cec36da4d) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:16.808543 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555fe137-82d0-46cb-9918-c231f25fed6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:16.810173 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555fe137-82d0-46cb-9918-c231f25fed6c None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:16.810799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-9045b85e-61d2-4440-9d4f-e347a0a0c662 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 08:47:16.810799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:16.810799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:16.810799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:16.810799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:16.810799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:16.810799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:16.810799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:16.810799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:16.810799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:16.810799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:16.810799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:16.810799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:16.810799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:16.810799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:16.810799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:16.811778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:16.811778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:16.811778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:16.811778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:16.811778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:16.811778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:16.811778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:16.811778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:16.811778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:16.811778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:16.811778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:16.811778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:16.811778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:16.811778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:16.811778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:16.811778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:16.811778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:16.811778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:16.812746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:16.812746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:16.812746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:16.812746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:16.812746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:16.812746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:16.812746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:16.812746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:16.812746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 08:47:16.812746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:16.812746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 345/687] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:47:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:16.813352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045b85e-61d2-4440-9d4f-e347a0a0c662 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:16.826603 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 343/688] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:47:16 2022] GET /identity/v3/role_assignments?user.id=ddf6f4d84bb34d2f810a0f6cec36da4d&scope.project.id=4227a5b804c0414c99d407bb1415cf49&role.id=391becda2bf04438b8d1763ab99acf1b => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:16.826963 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555fe137-82d0-46cb-9918-c231f25fed6c None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:16.982515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:17.004840 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9045b85e-61d2-4440-9d4f-e347a0a0c662 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:17.032549 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555fe137-82d0-46cb-9918-c231f25fed6c None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:17.045442 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eaaed0-55b4-473c-844d-f568928d81a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:17.046561 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eaaed0-55b4-473c-844d-f568928d81a4 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:17.046912 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eaaed0-55b4-473c-844d-f568928d81a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:17.047325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eaaed0-55b4-473c-844d-f568928d81a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:17.048176 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eaaed0-55b4-473c-844d-f568928d81a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:17.048654 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eaaed0-55b4-473c-844d-f568928d81a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:17.050147 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eaaed0-55b4-473c-844d-f568928d81a4 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:17.064792 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 346/689] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:17.070980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eaaed0-55b4-473c-844d-f568928d81a4 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:17.088643 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49befa-f761-45ca-9e74-14c100c0f1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:17.090114 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49befa-f761-45ca-9e74-14c100c0f1eb None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:17.090865 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49befa-f761-45ca-9e74-14c100c0f1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:17.091371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49befa-f761-45ca-9e74-14c100c0f1eb None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:17.093211 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49befa-f761-45ca-9e74-14c100c0f1eb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:17.093739 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49befa-f761-45ca-9e74-14c100c0f1eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:17.095432 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49befa-f761-45ca-9e74-14c100c0f1eb None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:17.119675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 344/690] 10.222.0.46 () {56 vars in 1135 bytes} [Tue Aug 2 08:47:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:17.124831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49befa-f761-45ca-9e74-14c100c0f1eb None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:17.181580 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:17.209376 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eaaed0-55b4-473c-844d-f568928d81a4 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:17.248134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49befa-f761-45ca-9e74-14c100c0f1eb None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:17.255110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860021fa-e75b-4444-a4b1-46b5feb3e3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:17.256525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860021fa-e75b-4444-a4b1-46b5feb3e3bf None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:17.256978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860021fa-e75b-4444-a4b1-46b5feb3e3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:17.257448 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860021fa-e75b-4444-a4b1-46b5feb3e3bf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:17.267963 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860021fa-e75b-4444-a4b1-46b5feb3e3bf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:17.270494 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860021fa-e75b-4444-a4b1-46b5feb3e3bf None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:17.288804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-860021fa-e75b-4444-a4b1-46b5feb3e3bf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:47:17.288804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:17.288804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:17.288804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:17.288804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:17.288804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:17.288804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:17.288804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:17.288804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:17.288804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:17.288804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:17.288804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:17.288804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:17.288804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:17.288804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:17.288804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:17.289949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:17.289949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:17.289949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:17.289949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:17.289949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:17.289949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:17.289949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:17.289949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:17.289949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:17.289949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:17.289949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:17.289949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:17.289949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:17.289949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:17.289949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:17.289949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:17.289949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:17.289949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:17.290926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:17.290926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:17.290926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:17.290926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:17.290926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:17.290926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:17.290926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:17.290926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:17.290926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:47:17.290926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:17.292516 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4834ca-b7f4-4263-8330-e513e16a377c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:17.293210 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4834ca-b7f4-4263-8330-e513e16a377c None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:17.293741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4834ca-b7f4-4263-8330-e513e16a377c None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:17.294177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4834ca-b7f4-4263-8330-e513e16a377c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:17.294377 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 347/691] 10.222.0.46 () {56 vars in 1093 bytes} [Tue Aug 2 08:47:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:17.307042 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860021fa-e75b-4444-a4b1-46b5feb3e3bf None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:17.330249 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:17.393298 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4834ca-b7f4-4263-8330-e513e16a377c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:17.394984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4834ca-b7f4-4263-8330-e513e16a377c None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:17.452528 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:17.515459 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860021fa-e75b-4444-a4b1-46b5feb3e3bf None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:17.531773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 345/692] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:47:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:47:17.540241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4834ca-b7f4-4263-8330-e513e16a377c None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:17.586767 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecfb6e-c91f-41e5-a473-d7e20a35d54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:17.588895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecfb6e-c91f-41e5-a473-d7e20a35d54f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:17.590075 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecfb6e-c91f-41e5-a473-d7e20a35d54f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:17.590692 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecfb6e-c91f-41e5-a473-d7e20a35d54f None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:17.592299 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ecfb6e-c91f-41e5-a473-d7e20a35d54f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:17.592969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ecfb6e-c91f-41e5-a473-d7e20a35d54f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:17.595811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ecfb6e-c91f-41e5-a473-d7e20a35d54f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:17.633467 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 348/693] 10.222.0.46 () {56 vars in 1133 bytes} [Tue Aug 2 08:47:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:17.637579 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecfb6e-c91f-41e5-a473-d7e20a35d54f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:17.705322 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4834ca-b7f4-4263-8330-e513e16a377c None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:17.712049 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:17.752484 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c51f8-8ae4-45bd-8799-9547c3c1b485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:17.753156 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c51f8-8ae4-45bd-8799-9547c3c1b485 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:17.753437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c51f8-8ae4-45bd-8799-9547c3c1b485 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:17.753713 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c51f8-8ae4-45bd-8799-9547c3c1b485 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:17.760251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecfb6e-c91f-41e5-a473-d7e20a35d54f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:17.763187 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:17.765577 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c51f8-8ae4-45bd-8799-9547c3c1b485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:17.766930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c51f8-8ae4-45bd-8799-9547c3c1b485 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:17.782691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-9f2c51f8-8ae4-45bd-8799-9547c3c1b485 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:17.782691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:17.782691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:17.782691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:17.782691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:17.782691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:17.782691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:17.782691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:17.782691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:17.782691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:17.782691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:17.782691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:17.782691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:17.782691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:17.782691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:17.783699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:17.783699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:17.783699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:17.783699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:17.783699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:17.783699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:17.783699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:17.783699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:17.783699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:17.783699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:17.783699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:17.783699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:17.783699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:17.783699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:17.783699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:17.783699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:17.783699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:17.784673 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:17.784673 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:17.784673 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:17.784673 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:17.784673 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 346/694] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:17.789458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b9750-4e98-48c2-8a83-e7741a134873 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:17.789458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b9750-4e98-48c2-8a83-e7741a134873 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:17.789458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b9750-4e98-48c2-8a83-e7741a134873 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:17.789458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 347/695] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:17.790268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b9750-4e98-48c2-8a83-e7741a134873 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:17.814943 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52690ed-d93e-4111-9cd4-ae92a4b0ef76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:17.816539 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52690ed-d93e-4111-9cd4-ae92a4b0ef76 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:17.817228 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52690ed-d93e-4111-9cd4-ae92a4b0ef76 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:17.817660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52690ed-d93e-4111-9cd4-ae92a4b0ef76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:17.907667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b9750-4e98-48c2-8a83-e7741a134873 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:17.915640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52690ed-d93e-4111-9cd4-ae92a4b0ef76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:17.917104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52690ed-d93e-4111-9cd4-ae92a4b0ef76 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:17.956570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4849ad-f0f7-4b91-8585-ca6125bd9562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:17.957787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4849ad-f0f7-4b91-8585-ca6125bd9562 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:17.959340 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4849ad-f0f7-4b91-8585-ca6125bd9562 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:17.959858 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4849ad-f0f7-4b91-8585-ca6125bd9562 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:17.960768 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4849ad-f0f7-4b91-8585-ca6125bd9562 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:17.961297 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4849ad-f0f7-4b91-8585-ca6125bd9562 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:17.962917 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4849ad-f0f7-4b91-8585-ca6125bd9562 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:17.985599 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 348/696] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:17.985599 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90128d-46bd-48a2-9bb9-2f7f100c9ff4 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:17.985599 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90128d-46bd-48a2-9bb9-2f7f100c9ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:17.985599 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc90128d-46bd-48a2-9bb9-2f7f100c9ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:17.996621 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:18.006693 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-cc90128d-46bd-48a2-9bb9-2f7f100c9ff4 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:18.008003 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-cc90128d-46bd-48a2-9bb9-2f7f100c9ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:18.049076 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 349/697] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:47:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:47:18.050278 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52690ed-d93e-4111-9cd4-ae92a4b0ef76 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:18.064955 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:18.086727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 349/698] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:47:18.087798 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90128d-46bd-48a2-9bb9-2f7f100c9ff4 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:18.188585 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52690ed-d93e-4111-9cd4-ae92a4b0ef76 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:18.205318 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc90128d-46bd-48a2-9bb9-2f7f100c9ff4 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:18.235455 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a414f44-0fcd-47d4-8996-a698d8ce8c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:18.236647 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a414f44-0fcd-47d4-8996-a698d8ce8c3b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:18.237385 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a414f44-0fcd-47d4-8996-a698d8ce8c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:18.237876 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a414f44-0fcd-47d4-8996-a698d8ce8c3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:18.243394 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5460bfd-82b7-4ad6-974b-335324c35905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:18.244128 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5460bfd-82b7-4ad6-974b-335324c35905 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:18.244483 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5460bfd-82b7-4ad6-974b-335324c35905 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:18.244831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5460bfd-82b7-4ad6-974b-335324c35905 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:18.248011 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a414f44-0fcd-47d4-8996-a698d8ce8c3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:18.249678 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a414f44-0fcd-47d4-8996-a698d8ce8c3b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:18.256394 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5460bfd-82b7-4ad6-974b-335324c35905 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:18.257985 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5460bfd-82b7-4ad6-974b-335324c35905 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:18.265832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-0a414f44-0fcd-47d4-8996-a698d8ce8c3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:18.265832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:18.265832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:18.265832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:18.265832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:18.265832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:18.265832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:18.265832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:18.265832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:18.265832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:18.265832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:18.265832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:18.265832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:18.265832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:18.265832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:18.266778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:18.266778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:18.266778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:18.266778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:18.266778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:18.266778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:18.266778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:18.266778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:18.266778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:18.266778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:18.266778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:18.266778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:18.266778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:18.266778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:18.266778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:18.266778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:18.266778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:18.267708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:18.267708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:18.267708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:18.267708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:18.267708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:18.267708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:18.267708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:18.267708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:18.275232 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 350/699] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:18.275232 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a414f44-0fcd-47d4-8996-a698d8ce8c3b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:18.276077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-c5460bfd-82b7-4ad6-974b-335324c35905 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:18.276077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:18.276077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:18.276077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:18.276077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:18.276077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:18.276077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:18.276077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:18.276077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:18.276077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:18.276077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:18.276077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:18.276077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:18.276077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:18.276077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:18.276942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:18.276942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:18.276942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:18.276942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:18.276942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:18.276942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:18.276942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:18.276942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:18.276942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:18.276942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:18.276942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:18.276942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:18.276942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:18.276942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:18.276942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:18.276942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:18.276942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:18.277846 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:18.277846 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:18.277846 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:18.277846 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:18.281247 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 350/700] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:18.282697 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5460bfd-82b7-4ad6-974b-335324c35905 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:18.318639 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:18.319995 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:18.392928 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5460bfd-82b7-4ad6-974b-335324c35905 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:18.407519 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a414f44-0fcd-47d4-8996-a698d8ce8c3b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:18.430369 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759671a6-3f66-4b5a-bff5-33839aa1f110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:18.430977 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759671a6-3f66-4b5a-bff5-33839aa1f110 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:18.431257 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759671a6-3f66-4b5a-bff5-33839aa1f110 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:18.431529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759671a6-3f66-4b5a-bff5-33839aa1f110 None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:18.432270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759671a6-3f66-4b5a-bff5-33839aa1f110 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:18.432573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759671a6-3f66-4b5a-bff5-33839aa1f110 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:18.434008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759671a6-3f66-4b5a-bff5-33839aa1f110 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:18.445787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 351/701] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:18.446261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73675ef5-0031-41f6-bb1a-d3859a1e7b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:18.446593 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759671a6-3f66-4b5a-bff5-33839aa1f110 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:18.447430 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73675ef5-0031-41f6-bb1a-d3859a1e7b49 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:18.448145 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73675ef5-0031-41f6-bb1a-d3859a1e7b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:18.448602 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73675ef5-0031-41f6-bb1a-d3859a1e7b49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:18.461023 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73675ef5-0031-41f6-bb1a-d3859a1e7b49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:18.462709 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73675ef5-0031-41f6-bb1a-d3859a1e7b49 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:18.477979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-73675ef5-0031-41f6-bb1a-d3859a1e7b49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:18.477979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:18.477979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:18.477979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:18.477979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:18.477979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:18.477979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:18.477979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:18.477979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:18.477979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:18.477979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:18.477979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:18.477979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:18.477979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:18.477979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:18.478908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:18.478908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:18.478908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:18.478908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:18.478908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:18.478908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:18.478908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:18.478908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:18.478908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:18.478908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:18.478908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:18.478908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:18.478908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:18.478908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:18.478908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:18.478908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:18.478908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:18.479819 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:18.479819 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:18.479819 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:18.479819 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:18.479819 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 351/702] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:18.480498 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac99f3-5697-46dc-b47d-e6ef585086c9 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:18.480498 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac99f3-5697-46dc-b47d-e6ef585086c9 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:18.480777 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac99f3-5697-46dc-b47d-e6ef585086c9 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:18.486003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 352/703] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:18.486674 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac99f3-5697-46dc-b47d-e6ef585086c9 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:18.554550 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:18.561045 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759671a6-3f66-4b5a-bff5-33839aa1f110 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:18.572643 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:18.597766 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac99f3-5697-46dc-b47d-e6ef585086c9 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:18.601044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29698620-ff79-46f6-ac0a-33ccb9ca3929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:18.601608 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29698620-ff79-46f6-ac0a-33ccb9ca3929 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:18.601916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29698620-ff79-46f6-ac0a-33ccb9ca3929 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:18.602200 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29698620-ff79-46f6-ac0a-33ccb9ca3929 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:18.602896 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29698620-ff79-46f6-ac0a-33ccb9ca3929 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:18.603206 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29698620-ff79-46f6-ac0a-33ccb9ca3929 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:18.604574 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29698620-ff79-46f6-ac0a-33ccb9ca3929 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:18.622028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 352/704] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:18.622950 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29698620-ff79-46f6-ac0a-33ccb9ca3929 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:18.642349 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e7cec0-9885-4fa3-9b95-7d399b7efafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:18.643270 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7cec0-9885-4fa3-9b95-7d399b7efafe None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:18.643557 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7cec0-9885-4fa3-9b95-7d399b7efafe None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:18.643835 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e7cec0-9885-4fa3-9b95-7d399b7efafe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:18.644626 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7cec0-9885-4fa3-9b95-7d399b7efafe None admin] RBAC: Adding query filter params (role.id=391becda2bf04438b8d1763ab99acf1b, scope.project.id=4227a5b804c0414c99d407bb1415cf49, user.id=ff92ef3b9da74ccab3fbc43f852e3ad0) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:18.644941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7cec0-9885-4fa3-9b95-7d399b7efafe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:18.647813 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e7cec0-9885-4fa3-9b95-7d399b7efafe None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:18.658019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 353/705] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:47:18 2022] GET /identity/v3/role_assignments?user.id=ff92ef3b9da74ccab3fbc43f852e3ad0&scope.project.id=4227a5b804c0414c99d407bb1415cf49&role.id=391becda2bf04438b8d1763ab99acf1b => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:18.659302 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7af36a-f1ab-4a49-b8b2-63c49784157a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:18.659589 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7af36a-f1ab-4a49-b8b2-63c49784157a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:18.659867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7af36a-f1ab-4a49-b8b2-63c49784157a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:18.682643 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-ea7af36a-f1ab-4a49-b8b2-63c49784157a None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:18.683899 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ea7af36a-f1ab-4a49-b8b2-63c49784157a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:18.766161 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29698620-ff79-46f6-ac0a-33ccb9ca3929 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:18.773909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 354/706] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:47:18.773909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7af36a-f1ab-4a49-b8b2-63c49784157a None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:18.828197 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c65ea-636b-4cb8-add0-d11e8cfa89dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:18.829494 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c65ea-636b-4cb8-add0-d11e8cfa89dd None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:18.830027 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c65ea-636b-4cb8-add0-d11e8cfa89dd None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:18.830723 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c65ea-636b-4cb8-add0-d11e8cfa89dd None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:18.831546 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c65ea-636b-4cb8-add0-d11e8cfa89dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:18.831857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c65ea-636b-4cb8-add0-d11e8cfa89dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:18.833230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c65ea-636b-4cb8-add0-d11e8cfa89dd None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:18.847773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 353/707] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:18.849039 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c65ea-636b-4cb8-add0-d11e8cfa89dd None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:18.854841 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:18.866653 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:18.897985 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7af36a-f1ab-4a49-b8b2-63c49784157a None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:18.948183 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fea72-efcb-4169-ad45-267ac971b26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:18.948406 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fea72-efcb-4169-ad45-267ac971b26c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:18.948792 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fea72-efcb-4169-ad45-267ac971b26c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:18.949190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0fea72-efcb-4169-ad45-267ac971b26c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:18.959751 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0fea72-efcb-4169-ad45-267ac971b26c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:18.961457 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0fea72-efcb-4169-ad45-267ac971b26c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:18.969374 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c65ea-636b-4cb8-add0-d11e8cfa89dd None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:18.973352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-4b0fea72-efcb-4169-ad45-267ac971b26c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:18.973352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:18.973352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:18.973352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:18.973352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:18.973352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:18.973352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:18.973352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:18.973352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:18.973352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:18.973352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:18.973352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:18.973352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:18.973352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:18.973352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:18.974344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:18.974344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:18.974344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:18.974344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:18.974344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:18.974344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:18.974344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:18.974344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:18.974344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:18.974344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:18.974344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:18.974344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:18.974344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:18.974344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:18.974344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:18.974344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:18.974344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:18.975828 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:18.975828 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:18.975828 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:18.975828 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:18.975828 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:18.975828 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:18.975828 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:18.975828 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:18.977970 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 355/708] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:18.979087 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fea72-efcb-4169-ad45-267ac971b26c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:19.019470 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85faccb2-694f-4ed7-8081-924314deabd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:19.020113 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85faccb2-694f-4ed7-8081-924314deabd5 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:19.020398 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85faccb2-694f-4ed7-8081-924314deabd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:19.020677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85faccb2-694f-4ed7-8081-924314deabd5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:19.021311 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85faccb2-694f-4ed7-8081-924314deabd5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:19.023244 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85faccb2-694f-4ed7-8081-924314deabd5 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:19.034310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-85faccb2-694f-4ed7-8081-924314deabd5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 08:47:19.034310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:19.034310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:47:19.034310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:47:19.034310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:47:19.034310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:47:19.034310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:47:19.034310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:19.034310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 08:47:19.034310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:19.034310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:19.034310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:19.034310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:19.034310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:19.034310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:19.034310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:19.035330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:19.035330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:19.035330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:19.035330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:19.035330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:19.035330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:19.035330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:19.035330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:47:19.035330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:47:19.035330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:47:19.035330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:47:19.035330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:19.035330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:19.035330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:19.035330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:19.035330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:19.036191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:19.036191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:19.036191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:19.036191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:19.036191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:19.036191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:19.036191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:19.036191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:19.036191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:19.036191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:19.036191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:19.036191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:47:19.036191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:19.036191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 08:47:19.036191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:19.036191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 354/709] 10.222.0.46 () {56 vars in 1129 bytes} [Tue Aug 2 08:47:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:19.037119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85faccb2-694f-4ed7-8081-924314deabd5 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:19.105876 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0fea72-efcb-4169-ad45-267ac971b26c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:19.143418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:19.145403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:19.156528 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce611bd-e0c8-4dfd-96c6-1322da187cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:19.162988 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce611bd-e0c8-4dfd-96c6-1322da187cc9 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:19.162988 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce611bd-e0c8-4dfd-96c6-1322da187cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:19.162988 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce611bd-e0c8-4dfd-96c6-1322da187cc9 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:19.162988 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce611bd-e0c8-4dfd-96c6-1322da187cc9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:19.162988 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce611bd-e0c8-4dfd-96c6-1322da187cc9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:19.172098 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85faccb2-694f-4ed7-8081-924314deabd5 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:19.207764 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f32d9e-7960-46f3-9dd2-6eddfaae0066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:19.208387 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f32d9e-7960-46f3-9dd2-6eddfaae0066 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:19.208671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f32d9e-7960-46f3-9dd2-6eddfaae0066 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:19.209500 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f32d9e-7960-46f3-9dd2-6eddfaae0066 None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:19.210336 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f32d9e-7960-46f3-9dd2-6eddfaae0066 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:19.210682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f32d9e-7960-46f3-9dd2-6eddfaae0066 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:19.212114 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f32d9e-7960-46f3-9dd2-6eddfaae0066 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:19.229727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 355/710] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:19.229727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f32d9e-7960-46f3-9dd2-6eddfaae0066 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:19.282654 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 356/711] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 2 08:47:18 2022] POST /identity/v3/users => generated 254 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:47:19.284080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce611bd-e0c8-4dfd-96c6-1322da187cc9 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:19.358536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f32d9e-7960-46f3-9dd2-6eddfaae0066 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:19.398375 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e63e9-f835-44f3-a57d-2a18903fd0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:19.399162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e63e9-f835-44f3-a57d-2a18903fd0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:19.400295 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e63e9-f835-44f3-a57d-2a18903fd0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:19.400593 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e63e9-f835-44f3-a57d-2a18903fd0f7 None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:19.401289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e63e9-f835-44f3-a57d-2a18903fd0f7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:19.401599 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e63e9-f835-44f3-a57d-2a18903fd0f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:19.403049 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e63e9-f835-44f3-a57d-2a18903fd0f7 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:19.409757 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 356/712] 10.222.0.46 () {56 vars in 1124 bytes} [Tue Aug 2 08:47:19 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:19.411788 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf6dc4-1864-43aa-995a-0b8bec1cdf13 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:19.412586 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf6dc4-1864-43aa-995a-0b8bec1cdf13 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:19.412933 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf6dc4-1864-43aa-995a-0b8bec1cdf13 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:19.417982 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 357/713] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:19.418941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf6dc4-1864-43aa-995a-0b8bec1cdf13 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:19.422466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:19.443104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce611bd-e0c8-4dfd-96c6-1322da187cc9 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:19.494247 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87208d2-516a-4692-89cf-dd8204d87275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:19.494938 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87208d2-516a-4692-89cf-dd8204d87275 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:19.495618 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87208d2-516a-4692-89cf-dd8204d87275 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:19.495909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87208d2-516a-4692-89cf-dd8204d87275 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:19.497000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87208d2-516a-4692-89cf-dd8204d87275 None admin] RBAC: Adding query filter params (role.id=8ce0850e15804f3d811e88cc9841e673, scope.project.id=4227a5b804c0414c99d407bb1415cf49, user.id=3191d804e8dc4b1a85f490319833ec07) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:19.497595 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87208d2-516a-4692-89cf-dd8204d87275 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:19.501045 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87208d2-516a-4692-89cf-dd8204d87275 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:19.506966 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 357/714] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:47:19 2022] GET /identity/v3/role_assignments?user.id=3191d804e8dc4b1a85f490319833ec07&scope.project.id=4227a5b804c0414c99d407bb1415cf49&role.id=8ce0850e15804f3d811e88cc9841e673 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:19.508228 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e33419-058f-4221-9846-558bd37c8746 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:19.508515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e33419-058f-4221-9846-558bd37c8746 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:19.508795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e33419-058f-4221-9846-558bd37c8746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:19.528835 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-b3e33419-058f-4221-9846-558bd37c8746 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:19.530997 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b3e33419-058f-4221-9846-558bd37c8746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:19.563595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf6dc4-1864-43aa-995a-0b8bec1cdf13 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:19.591136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:19.612895 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b26e6-257e-4d85-bff6-ec8591a7cd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:19.613781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b26e6-257e-4d85-bff6-ec8591a7cd5d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:19.614476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b26e6-257e-4d85-bff6-ec8591a7cd5d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:19.614913 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b26e6-257e-4d85-bff6-ec8591a7cd5d None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:19.617055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b26e6-257e-4d85-bff6-ec8591a7cd5d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:19.617548 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b26e6-257e-4d85-bff6-ec8591a7cd5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:19.619141 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b26e6-257e-4d85-bff6-ec8591a7cd5d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:19.621375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 358/715] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:47:19.626302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 358/716] 10.222.0.46 () {56 vars in 1124 bytes} [Tue Aug 2 08:47:19 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:19.647731 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e33419-058f-4221-9846-558bd37c8746 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:19.783426 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e33419-058f-4221-9846-558bd37c8746 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:19.843195 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82033e-8130-4e6b-a669-2f7a7e150f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:19.844297 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82033e-8130-4e6b-a669-2f7a7e150f00 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:19.844790 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82033e-8130-4e6b-a669-2f7a7e150f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:19.845285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82033e-8130-4e6b-a669-2f7a7e150f00 None admin] PATH_INFO: `/v3/services/network` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:19.846197 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f82033e-8130-4e6b-a669-2f7a7e150f00 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:19.847812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f82033e-8130-4e6b-a669-2f7a7e150f00 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:19.866230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-5f82033e-8130-4e6b-a669-2f7a7e150f00 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 08:47:19.866230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:19.866230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:47:19.866230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:47:19.866230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:47:19.866230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:47:19.866230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:47:19.866230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:19.866230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 08:47:19.866230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:19.866230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:19.866230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:19.866230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:19.866230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:19.866230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:19.866230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:19.866230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:19.867658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:19.867658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:19.867658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:19.867658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:19.867658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:19.867658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:19.867658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:47:19.867658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:47:19.867658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:47:19.867658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:47:19.867658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:19.867658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:19.867658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:19.867658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:19.867658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:19.867658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:19.868553 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:19.868553 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:19.868553 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:19.868553 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:19.868553 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:19.868553 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:19.868553 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:19.868553 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:19.868553 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:19.868553 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:19.868553 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:47:19.868553 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:19.868553 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 08:47:19.868553 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:19.871402 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 359/717] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:19 2022] GET /identity/v3/services/network => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:19.877009 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b26e6-257e-4d85-bff6-ec8591a7cd5d None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:19.963293 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:20.013923 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b26e6-257e-4d85-bff6-ec8591a7cd5d None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:20.059946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11196889-05b4-413d-a82f-2254bb95d6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:20.060580 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11196889-05b4-413d-a82f-2254bb95d6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:20.060865 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11196889-05b4-413d-a82f-2254bb95d6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:20.061143 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11196889-05b4-413d-a82f-2254bb95d6f6 None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:20.061832 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11196889-05b4-413d-a82f-2254bb95d6f6 None admin] RBAC: Adding query filter params (name=network) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:20.062169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11196889-05b4-413d-a82f-2254bb95d6f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:20.063545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11196889-05b4-413d-a82f-2254bb95d6f6 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:20.074540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 359/718] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 2 08:47:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:20.080082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82033e-8130-4e6b-a669-2f7a7e150f00 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:20.139784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:20.216660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82033e-8130-4e6b-a669-2f7a7e150f00 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:20.259945 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e515ed4f-985e-473a-a6c0-0209ecddffda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:20.260962 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515ed4f-985e-473a-a6c0-0209ecddffda None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:20.261442 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515ed4f-985e-473a-a6c0-0209ecddffda None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:20.261944 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e515ed4f-985e-473a-a6c0-0209ecddffda None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:20.262826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e515ed4f-985e-473a-a6c0-0209ecddffda None admin] RBAC: Adding query filter params (type=network) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:20.263335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e515ed4f-985e-473a-a6c0-0209ecddffda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:20.264895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e515ed4f-985e-473a-a6c0-0209ecddffda None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:20.271154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 360/719] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 2 08:47:20 2022] GET /identity/v3/services?type=network => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:21.211395 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df29a26-db1b-432a-924c-5e887615f58b None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:21.211884 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df29a26-db1b-432a-924c-5e887615f58b None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:21.212179 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df29a26-db1b-432a-924c-5e887615f58b None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:21.213795 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 360/720] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:21.219185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83e2ac-0ef3-49e4-bd3c-353b002d7803 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:21.219605 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83e2ac-0ef3-49e4-bd3c-353b002d7803 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:21.219806 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83e2ac-0ef3-49e4-bd3c-353b002d7803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:21.244983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-1b83e2ac-0ef3-49e4-bd3c-353b002d7803 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:21.246220 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1b83e2ac-0ef3-49e4-bd3c-353b002d7803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:21.337007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 361/721] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:47:21.374138 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df29a26-db1b-432a-924c-5e887615f58b None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:21.523738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df29a26-db1b-432a-924c-5e887615f58b None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:21.562755 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:21.569170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ae2bb-2e01-43c4-91a7-3dc385e673f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:21.569812 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ae2bb-2e01-43c4-91a7-3dc385e673f0 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:21.570119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ae2bb-2e01-43c4-91a7-3dc385e673f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:21.570395 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ae2bb-2e01-43c4-91a7-3dc385e673f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:21.586761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5ae2bb-2e01-43c4-91a7-3dc385e673f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:47:21.591677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-ba5ae2bb-2e01-43c4-91a7-3dc385e673f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:47:21.591677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:21.591677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:21.591677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:21.591677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:21.591677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:21.591677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:21.591677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:21.591677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:21.591677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:21.591677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:21.591677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:21.591677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:47:21.591677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:47:21.591677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:47:21.591677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:47:21.592614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:47:21.592614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:47:21.592614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:47:21.592614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:47:21.592614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:21.592614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:21.592614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:21.592614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:21.592614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:21.592614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:21.592614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:21.592614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:21.592614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:21.592614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:21.592614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:21.592614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:21.593871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:21.593871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:21.593871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:21.593871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:21.593871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:47:21.593871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:47:21.593871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:47:21.593871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:47:21.593871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:47:21.593871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:47:21.593871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:47:21.593871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:21.593871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 361/722] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:47:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:21.601311 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83e2ac-0ef3-49e4-bd3c-353b002d7803 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:21.655015 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:21.741487 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83e2ac-0ef3-49e4-bd3c-353b002d7803 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:21.790469 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725a495-6454-4782-9f39-709388e518d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:21.791014 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725a495-6454-4782-9f39-709388e518d9 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:21.791298 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725a495-6454-4782-9f39-709388e518d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:21.791573 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725a495-6454-4782-9f39-709388e518d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:21.792382 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c725a495-6454-4782-9f39-709388e518d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:21.792688 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c725a495-6454-4782-9f39-709388e518d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:21.794090 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c725a495-6454-4782-9f39-709388e518d9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:21.807228 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 362/723] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:47:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:21.812856 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ae2bb-2e01-43c4-91a7-3dc385e673f0 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:21.967230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b634931-2899-4960-a83f-2f3207022575 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:21.967547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b634931-2899-4960-a83f-2f3207022575 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:21.967798 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b634931-2899-4960-a83f-2f3207022575 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:21.969026 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5ae2bb-2e01-43c4-91a7-3dc385e673f0 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:21.969389 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 363/724] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:21.975311 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb50d5-9b4b-49e7-93e4-872feb0de9f0 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:21.975596 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb50d5-9b4b-49e7-93e4-872feb0de9f0 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:21.975877 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb50d5-9b4b-49e7-93e4-872feb0de9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:22.007336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-fceb50d5-9b4b-49e7-93e4-872feb0de9f0 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:22.009507 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-fceb50d5-9b4b-49e7-93e4-872feb0de9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:22.032400 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce2d29c-178f-4b8f-8586-b30a67143de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:22.033468 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2d29c-178f-4b8f-8586-b30a67143de3 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:22.034008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2d29c-178f-4b8f-8586-b30a67143de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:22.034497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2d29c-178f-4b8f-8586-b30a67143de3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:22.057261 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce2d29c-178f-4b8f-8586-b30a67143de3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:22.059096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce2d29c-178f-4b8f-8586-b30a67143de3 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:22.087443 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-6ce2d29c-178f-4b8f-8586-b30a67143de3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:22.087443 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:22.087443 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:22.087443 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:22.087443 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:22.087443 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:22.087443 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:22.087443 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:22.087443 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:22.087443 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:22.087443 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:22.087443 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:22.087443 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:22.087443 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:22.087443 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:22.088456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:22.088456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:22.088456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:22.088456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:22.088456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:22.088456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:22.088456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:22.088456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:22.088456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:22.088456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:22.088456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:22.088456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:22.088456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:22.088456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:22.088456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:22.088456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:22.088456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:22.089398 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:22.089398 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:22.089398 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:22.089398 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:22.094024 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 362/725] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:22.101861 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce2d29c-178f-4b8f-8586-b30a67143de3 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:22.140743 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 364/726] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 08:47:22.168641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb50d5-9b4b-49e7-93e4-872feb0de9f0 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:22.182672 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:22.219944 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:22.287961 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce2d29c-178f-4b8f-8586-b30a67143de3 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:22.308461 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb50d5-9b4b-49e7-93e4-872feb0de9f0 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:22.329579 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3e2d17-a788-4f05-ab94-8de1b846fd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:22.330220 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e2d17-a788-4f05-ab94-8de1b846fd20 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:22.330500 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e2d17-a788-4f05-ab94-8de1b846fd20 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:22.331200 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3e2d17-a788-4f05-ab94-8de1b846fd20 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:22.332165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e2d17-a788-4f05-ab94-8de1b846fd20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:22.332477 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e2d17-a788-4f05-ab94-8de1b846fd20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:22.334879 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3e2d17-a788-4f05-ab94-8de1b846fd20 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:22.347918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 363/727] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:22.348625 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280ebcf-6b00-4cc9-bb5a-f4db4c1e76cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:22.349331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b03372-4ae2-4803-b9e1-351a11b04fbc None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:22.349792 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280ebcf-6b00-4cc9-bb5a-f4db4c1e76cb None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:22.349930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b03372-4ae2-4803-b9e1-351a11b04fbc None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:22.350368 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280ebcf-6b00-4cc9-bb5a-f4db4c1e76cb None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:22.350493 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b03372-4ae2-4803-b9e1-351a11b04fbc None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:22.350921 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280ebcf-6b00-4cc9-bb5a-f4db4c1e76cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:22.352247 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 364/728] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:22.357354 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86045077-4e85-476e-8fb1-ed230de80b21 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:22.357648 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86045077-4e85-476e-8fb1-ed230de80b21 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:22.357942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86045077-4e85-476e-8fb1-ed230de80b21 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:22.359382 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 365/729] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:22.360984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914d0fd-331e-42d1-9316-664c572a053d None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:22.361480 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914d0fd-331e-42d1-9316-664c572a053d None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:22.362015 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914d0fd-331e-42d1-9316-664c572a053d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:22.370907 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280ebcf-6b00-4cc9-bb5a-f4db4c1e76cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:47:22.377285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-9280ebcf-6b00-4cc9-bb5a-f4db4c1e76cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:22.377285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:22.377285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:22.377285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:22.377285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:22.377285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:22.377285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:22.377285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:22.377285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:22.377285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:22.377285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:22.377285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:22.377285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:47:22.377285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:47:22.377285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:47:22.377285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:47:22.378795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:47:22.378795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:47:22.378795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:47:22.378795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:47:22.378795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:22.378795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:22.378795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:22.378795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:22.378795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:22.378795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:22.378795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:22.378795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:22.378795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:22.378795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:22.378795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:22.378795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:22.379641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:22.379641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:22.379641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:22.379641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:22.379641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:47:22.379641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:47:22.379641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:47:22.379641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:47:22.379641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:47:22.379641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:47:22.379641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:22.379641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:22.379641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 365/730] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 2 08:47:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:22.379641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280ebcf-6b00-4cc9-bb5a-f4db4c1e76cb None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:22.387612 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-f914d0fd-331e-42d1-9316-664c572a053d None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:22.388940 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f914d0fd-331e-42d1-9316-664c572a053d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:22.458743 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 366/731] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:47:22.463268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eea969-ecbc-4505-9466-ebf6e863f667 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:22.463562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eea969-ecbc-4505-9466-ebf6e863f667 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:22.463846 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eea969-ecbc-4505-9466-ebf6e863f667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:22.474965 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:22.482870 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-64eea969-ecbc-4505-9466-ebf6e863f667 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:22.484068 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-64eea969-ecbc-4505-9466-ebf6e863f667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:22.488250 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280ebcf-6b00-4cc9-bb5a-f4db4c1e76cb None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:22.512225 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:22.524935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584566e-8512-44a0-8750-47e62b3deee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:22.525600 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584566e-8512-44a0-8750-47e62b3deee0 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:22.525869 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584566e-8512-44a0-8750-47e62b3deee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:22.526186 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9584566e-8512-44a0-8750-47e62b3deee0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:22.533769 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584566e-8512-44a0-8750-47e62b3deee0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:22.535239 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9584566e-8512-44a0-8750-47e62b3deee0 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:22.547593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-9584566e-8512-44a0-8750-47e62b3deee0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 08:47:22.547593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:22.547593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:22.547593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:22.547593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:22.547593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:22.547593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:22.547593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:22.547593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:22.547593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:22.547593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:22.547593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:22.547593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:22.547593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:22.547593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:22.547593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:22.548540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:22.548540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:22.548540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:22.548540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:22.548540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:22.548540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:22.548540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:22.548540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:22.548540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:22.548540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:22.548540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:22.548540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:22.548540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:22.548540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:22.548540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:22.548540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:22.548540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:22.548540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:22.549499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:22.549499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:22.549499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:22.549499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:22.549499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:22.549499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:22.549499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:22.549499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:22.549499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 08:47:22.549499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:22.553525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 366/732] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:47:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:22.554379 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584566e-8512-44a0-8750-47e62b3deee0 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:22.566256 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 367/733] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:47:22.567763 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eea969-ecbc-4505-9466-ebf6e863f667 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:22.664055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9584566e-8512-44a0-8750-47e62b3deee0 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:22.673484 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eea969-ecbc-4505-9466-ebf6e863f667 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:22.699827 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d3bdd3-1fc9-4c0a-b56f-e17d69eb5241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:22.700450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3bdd3-1fc9-4c0a-b56f-e17d69eb5241 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:22.700733 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3bdd3-1fc9-4c0a-b56f-e17d69eb5241 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:22.701010 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d3bdd3-1fc9-4c0a-b56f-e17d69eb5241 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:22.701728 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d3bdd3-1fc9-4c0a-b56f-e17d69eb5241 None admin] RBAC: Adding query filter params (name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:22.702066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d3bdd3-1fc9-4c0a-b56f-e17d69eb5241 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:22.703429 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d3bdd3-1fc9-4c0a-b56f-e17d69eb5241 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:22.709999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f1ca7-5291-4764-90a2-997d3a8a6b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:22.710647 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f1ca7-5291-4764-90a2-997d3a8a6b06 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:22.710933 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f1ca7-5291-4764-90a2-997d3a8a6b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:22.711212 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f1ca7-5291-4764-90a2-997d3a8a6b06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:22.717153 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 367/734] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 2 08:47:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:22.717153 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d3bdd3-1fc9-4c0a-b56f-e17d69eb5241 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:22.725914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f1ca7-5291-4764-90a2-997d3a8a6b06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:47:22.731274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-767f1ca7-5291-4764-90a2-997d3a8a6b06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:47:22.731274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:22.731274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:22.731274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:22.731274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:22.731274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:22.731274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:22.731274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:22.731274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:22.731274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:22.731274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:22.731274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:22.731274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:47:22.731274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:47:22.731274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:47:22.731274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:47:22.732365 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:47:22.732365 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:47:22.732365 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:47:22.732365 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:47:22.732365 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:22.732365 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:22.732365 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:22.732365 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:22.732365 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:22.732365 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:22.732365 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:22.732365 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:22.732365 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:22.732365 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:22.732365 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:22.732365 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:22.733203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:22.733203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:22.733203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:22.733203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:22.733203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:47:22.733203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:47:22.733203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:47:22.733203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:47:22.733203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:47:22.733203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:47:22.733203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:47:22.733203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:22.733203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 368/735] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:47:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:22.734000 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f1ca7-5291-4764-90a2-997d3a8a6b06 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:22.735221 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:22.760478 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:22.826364 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d3bdd3-1fc9-4c0a-b56f-e17d69eb5241 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:22.838409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f1ca7-5291-4764-90a2-997d3a8a6b06 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:22.859790 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3040b0-15fe-4c9a-8aa0-2062af6b70b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:22.862444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3040b0-15fe-4c9a-8aa0-2062af6b70b1 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:22.862444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3040b0-15fe-4c9a-8aa0-2062af6b70b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:22.862444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3040b0-15fe-4c9a-8aa0-2062af6b70b1 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:22.862444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3040b0-15fe-4c9a-8aa0-2062af6b70b1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:22.862444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3040b0-15fe-4c9a-8aa0-2062af6b70b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:22.863540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3040b0-15fe-4c9a-8aa0-2062af6b70b1 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:22.872727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51fdff-22b4-4713-9c31-c967c08f95e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:22.873349 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51fdff-22b4-4713-9c31-c967c08f95e6 None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:22.873630 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51fdff-22b4-4713-9c31-c967c08f95e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:22.874011 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51fdff-22b4-4713-9c31-c967c08f95e6 None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:22.874706 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51fdff-22b4-4713-9c31-c967c08f95e6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:22.876069 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b51fdff-22b4-4713-9c31-c967c08f95e6 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:22.886478 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 368/736] 10.222.0.46 () {56 vars in 1135 bytes} [Tue Aug 2 08:47:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:22.886478 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3040b0-15fe-4c9a-8aa0-2062af6b70b1 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:22.947005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:22.979040 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3040b0-15fe-4c9a-8aa0-2062af6b70b1 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:22.981542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 369/737] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 2 08:47:22 2022] POST /identity/v3/services => generated 251 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:47:22.982620 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51fdff-22b4-4713-9c31-c967c08f95e6 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:23.025640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58db1a60-b168-445a-a0a0-f76ca2dd8f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:23.026339 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db1a60-b168-445a-a0a0-f76ca2dd8f23 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:23.026623 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db1a60-b168-445a-a0a0-f76ca2dd8f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:23.026899 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58db1a60-b168-445a-a0a0-f76ca2dd8f23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:23.041296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db1a60-b168-445a-a0a0-f76ca2dd8f23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:23.042814 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58db1a60-b168-445a-a0a0-f76ca2dd8f23 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:23.060151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-58db1a60-b168-445a-a0a0-f76ca2dd8f23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:23.060151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:23.060151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:23.060151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:23.060151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:23.060151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:23.060151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:23.060151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:23.060151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:23.060151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:23.060151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:23.060151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:23.060151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:23.060151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:23.060151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:23.061149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:23.061149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:23.061149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:23.061149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:23.061149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:23.061149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:23.061149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:23.061149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:23.061149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:23.061149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:23.061149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:23.061149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:23.061149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:23.061149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:23.061149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:23.061149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:23.061149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:23.062102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:23.062102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:23.062102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:23.062102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:23.062102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 369/738] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:23.062933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b969147-eb1d-4d51-808b-1b4522234717 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:23.062933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b969147-eb1d-4d51-808b-1b4522234717 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:23.063185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b969147-eb1d-4d51-808b-1b4522234717 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:23.064673 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 370/739] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:23.065216 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b969147-eb1d-4d51-808b-1b4522234717 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:23.090225 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:23.112515 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51fdff-22b4-4713-9c31-c967c08f95e6 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:23.150967 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88b226-9064-4d24-8497-46608cd7d652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:23.151979 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88b226-9064-4d24-8497-46608cd7d652 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:23.152407 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88b226-9064-4d24-8497-46608cd7d652 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:23.152846 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88b226-9064-4d24-8497-46608cd7d652 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:23.153721 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa88b226-9064-4d24-8497-46608cd7d652 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:23.154066 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa88b226-9064-4d24-8497-46608cd7d652 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:23.155533 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa88b226-9064-4d24-8497-46608cd7d652 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:23.165568 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 370/740] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:47:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:23.166390 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88b226-9064-4d24-8497-46608cd7d652 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:23.184215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b969147-eb1d-4d51-808b-1b4522234717 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:23.202998 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:23.221642 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1aba5-a7c6-40c5-aaf7-d7e4bdc26f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:23.222400 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1aba5-a7c6-40c5-aaf7-d7e4bdc26f7b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:23.222619 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1aba5-a7c6-40c5-aaf7-d7e4bdc26f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:23.222893 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de1aba5-a7c6-40c5-aaf7-d7e4bdc26f7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:23.280857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88b226-9064-4d24-8497-46608cd7d652 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:23.295507 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de1aba5-a7c6-40c5-aaf7-d7e4bdc26f7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:23.297575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de1aba5-a7c6-40c5-aaf7-d7e4bdc26f7b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:23.317818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11002ec2-3870-4dfb-a9c5-bba60d19e039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:23.319144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11002ec2-3870-4dfb-a9c5-bba60d19e039 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:23.319430 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11002ec2-3870-4dfb-a9c5-bba60d19e039 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:23.319707 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11002ec2-3870-4dfb-a9c5-bba60d19e039 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:23.320470 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11002ec2-3870-4dfb-a9c5-bba60d19e039 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:23.320783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11002ec2-3870-4dfb-a9c5-bba60d19e039 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:23.322471 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11002ec2-3870-4dfb-a9c5-bba60d19e039 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:23.337768 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 371/741] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:23.343355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e35af-9460-4ae7-aeb1-a7dbcd1fa009 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:23.343590 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e35af-9460-4ae7-aeb1-a7dbcd1fa009 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:23.343973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e35af-9460-4ae7-aeb1-a7dbcd1fa009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:23.353440 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:23.364486 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-db9e35af-9460-4ae7-aeb1-a7dbcd1fa009 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:23.366539 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-db9e35af-9460-4ae7-aeb1-a7dbcd1fa009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:23.406176 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 371/742] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:47:23 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 08:47:23.407027 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1aba5-a7c6-40c5-aaf7-d7e4bdc26f7b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:23.427717 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:23.438501 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 372/743] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:23 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 08:47:23.439352 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e35af-9460-4ae7-aeb1-a7dbcd1fa009 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:23.528758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de1aba5-a7c6-40c5-aaf7-d7e4bdc26f7b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:23.554069 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e35af-9460-4ae7-aeb1-a7dbcd1fa009 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:23.564555 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba6874-53dc-4d8f-9657-ed156b96f61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:23.565509 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba6874-53dc-4d8f-9657-ed156b96f61f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:23.565797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba6874-53dc-4d8f-9657-ed156b96f61f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:23.566113 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba6874-53dc-4d8f-9657-ed156b96f61f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:23.577425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba6874-53dc-4d8f-9657-ed156b96f61f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:23.578912 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba6874-53dc-4d8f-9657-ed156b96f61f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:23.590478 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f467c-465a-46c2-bec8-6f8ba47aca2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:23.591418 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f467c-465a-46c2-bec8-6f8ba47aca2d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:23.591888 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f467c-465a-46c2-bec8-6f8ba47aca2d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:23.592348 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502f467c-465a-46c2-bec8-6f8ba47aca2d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:23.594798 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:23.594918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-2bba6874-53dc-4d8f-9657-ed156b96f61f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:23.594918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:23.594918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:23.594918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:23.594918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:23.594918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:23.594918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:23.594918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:23.594918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:23.594918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:23.594918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:23.594918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:23.594918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:23.594918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:23.594918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:23.595757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:23.595757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:23.595757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:23.595757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:23.595757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:23.595757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:23.595757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:23.595757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:23.595757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:23.595757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:23.595757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:23.595757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:23.595757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:23.595757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:23.595757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:23.595757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:23.595757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:23.596660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:23.596660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:23.596660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:23.596660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:23.596660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 372/744] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:23.597935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba6874-53dc-4d8f-9657-ed156b96f61f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:23.605810 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502f467c-465a-46c2-bec8-6f8ba47aca2d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:23.607381 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502f467c-465a-46c2-bec8-6f8ba47aca2d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:23.621754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-502f467c-465a-46c2-bec8-6f8ba47aca2d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 08:47:23.621754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:23.621754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:23.621754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:23.621754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:23.621754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:23.621754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:23.621754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:23.621754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:23.621754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:23.621754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:23.621754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:23.621754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:23.621754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:23.621754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:23.621754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:23.622830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:23.622830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:23.622830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:23.622830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:23.622830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:23.622830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:23.622830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:23.622830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:23.622830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:23.622830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:23.622830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:23.622830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:23.622830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:23.622830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:23.622830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:23.622830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:23.622830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:23.622830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:23.624446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:23.624446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:23.624446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:23.624446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:23.624446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:23.624446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:23.624446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:23.624446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:23.624446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 08:47:23.624446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:23.624446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 373/745] 10.222.0.46 () {56 vars in 1117 bytes} [Tue Aug 2 08:47:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:23.624446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f467c-465a-46c2-bec8-6f8ba47aca2d None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:23.743877 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502f467c-465a-46c2-bec8-6f8ba47aca2d None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:23.784489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4a961-04aa-4122-aefa-38964256b73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:23.785104 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4a961-04aa-4122-aefa-38964256b73f None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:23.785385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4a961-04aa-4122-aefa-38964256b73f None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:23.785657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4a961-04aa-4122-aefa-38964256b73f None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:23.786509 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4a961-04aa-4122-aefa-38964256b73f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:23.790271 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c4a961-04aa-4122-aefa-38964256b73f None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:23.795244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:23.828258 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba6874-53dc-4d8f-9657-ed156b96f61f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:23.861355 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158ea99-bcc0-4d0f-be90-d0ae58f3b949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:23.861948 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158ea99-bcc0-4d0f-be90-d0ae58f3b949 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:23.862237 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158ea99-bcc0-4d0f-be90-d0ae58f3b949 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:23.862511 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158ea99-bcc0-4d0f-be90-d0ae58f3b949 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:23.872437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158ea99-bcc0-4d0f-be90-d0ae58f3b949 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:23.874352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f158ea99-bcc0-4d0f-be90-d0ae58f3b949 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:23.885909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 374/746] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 2 08:47:23 2022] POST /identity/v3/services => generated 240 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:47:23.885909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4a961-04aa-4122-aefa-38964256b73f None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:23.896450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-f158ea99-bcc0-4d0f-be90-d0ae58f3b949 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:23.896450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:23.896450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:23.896450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:23.896450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:23.896450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:23.896450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:23.896450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:23.896450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:23.896450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:23.896450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:23.896450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:23.896450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:23.896450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:23.896450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:23.898092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:23.898092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:23.898092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:23.898092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:23.898092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:23.898092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:23.898092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:23.898092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:23.898092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:23.898092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:23.898092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:23.898092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:23.898092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:23.898092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:23.898092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:23.898092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:23.898092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:23.899066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:23.899066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:23.899066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:23.899066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:23.902075 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 373/747] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:23.903351 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158ea99-bcc0-4d0f-be90-d0ae58f3b949 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:23.983817 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:24.038474 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158ea99-bcc0-4d0f-be90-d0ae58f3b949 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:24.054586 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4a961-04aa-4122-aefa-38964256b73f None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:24.063063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:24.080171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a97794-ba07-4d72-83ea-e2c1c05642fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:24.080804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a97794-ba07-4d72-83ea-e2c1c05642fa None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:24.081089 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a97794-ba07-4d72-83ea-e2c1c05642fa None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:24.081363 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a97794-ba07-4d72-83ea-e2c1c05642fa None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:24.082437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a97794-ba07-4d72-83ea-e2c1c05642fa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:24.082575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a97794-ba07-4d72-83ea-e2c1c05642fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:24.084080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a97794-ba07-4d72-83ea-e2c1c05642fa None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:24.100814 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649d312-2f07-4458-b503-584d461d9b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:24.101585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649d312-2f07-4458-b503-584d461d9b1e None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:24.102011 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649d312-2f07-4458-b503-584d461d9b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:24.102408 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649d312-2f07-4458-b503-584d461d9b1e None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:24.103225 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649d312-2f07-4458-b503-584d461d9b1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:24.103640 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649d312-2f07-4458-b503-584d461d9b1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:24.105238 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649d312-2f07-4458-b503-584d461d9b1e None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:24.106868 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 374/748] 10.222.0.46 () {56 vars in 1149 bytes} [Tue Aug 2 08:47:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:24.107758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a97794-ba07-4d72-83ea-e2c1c05642fa None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:24.119824 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 375/749] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:24.120837 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649d312-2f07-4458-b503-584d461d9b1e None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:24.240858 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a97794-ba07-4d72-83ea-e2c1c05642fa None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:24.247737 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649d312-2f07-4458-b503-584d461d9b1e None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:24.262041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:24.285717 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41130b8c-623c-4c22-9fb4-440aff914d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:24.286576 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41130b8c-623c-4c22-9fb4-440aff914d84 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:24.287089 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41130b8c-623c-4c22-9fb4-440aff914d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:24.292084 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41130b8c-623c-4c22-9fb4-440aff914d84 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:24.292084 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41130b8c-623c-4c22-9fb4-440aff914d84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:24.292084 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41130b8c-623c-4c22-9fb4-440aff914d84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:24.292084 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41130b8c-623c-4c22-9fb4-440aff914d84 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:24.297038 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c296f7e-c8e1-4a90-95a8-b5a952dfd169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:24.298440 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c296f7e-c8e1-4a90-95a8-b5a952dfd169 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:24.298901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c296f7e-c8e1-4a90-95a8-b5a952dfd169 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:24.299324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c296f7e-c8e1-4a90-95a8-b5a952dfd169 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:24.308686 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 375/750] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:24.309723 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41130b8c-623c-4c22-9fb4-440aff914d84 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:24.367164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:24.383171 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c296f7e-c8e1-4a90-95a8-b5a952dfd169 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:24.384698 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c296f7e-c8e1-4a90-95a8-b5a952dfd169 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:24.453097 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41130b8c-623c-4c22-9fb4-440aff914d84 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:24.501494 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 376/751] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:47:24 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:47:24.501494 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c296f7e-c8e1-4a90-95a8-b5a952dfd169 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:24.507316 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfa84f-4c6d-478c-883a-6777df43ec32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:24.508187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfa84f-4c6d-478c-883a-6777df43ec32 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:24.509156 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfa84f-4c6d-478c-883a-6777df43ec32 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:24.509596 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dfa84f-4c6d-478c-883a-6777df43ec32 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:24.513011 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:24.520315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dfa84f-4c6d-478c-883a-6777df43ec32 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:24.522350 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dfa84f-4c6d-478c-883a-6777df43ec32 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:24.539007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-19dfa84f-4c6d-478c-883a-6777df43ec32 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:47:24.539007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:24.539007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:24.539007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:24.539007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:24.539007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:24.539007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:24.539007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:24.539007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:24.539007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:24.539007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:24.539007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:24.539007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:24.539007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:24.539007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:24.539007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:24.544764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:24.544764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:24.544764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:24.544764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:24.544764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:24.544764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:24.544764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:24.544764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:24.544764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:24.544764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:24.544764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:24.544764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:24.544764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:24.544764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:24.544764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:24.544764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:24.544764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:24.544764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:24.546375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:24.546375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:24.546375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:24.546375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:24.546375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:24.546375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:24.546375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:24.546375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:24.546375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:47:24.546375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:24.546375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 376/752] 10.222.0.46 () {56 vars in 1093 bytes} [Tue Aug 2 08:47:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:24.546375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfa84f-4c6d-478c-883a-6777df43ec32 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:24.622995 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c296f7e-c8e1-4a90-95a8-b5a952dfd169 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:24.669759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35674e93-156f-4c8c-a762-0437fc95ae36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:24.670461 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35674e93-156f-4c8c-a762-0437fc95ae36 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:24.670744 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35674e93-156f-4c8c-a762-0437fc95ae36 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:24.671396 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:24.671510 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35674e93-156f-4c8c-a762-0437fc95ae36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:24.676526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dfa84f-4c6d-478c-883a-6777df43ec32 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:24.680598 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35674e93-156f-4c8c-a762-0437fc95ae36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:24.682080 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35674e93-156f-4c8c-a762-0437fc95ae36 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:24.697123 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-35674e93-156f-4c8c-a762-0437fc95ae36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:24.697123 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:24.697123 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:24.697123 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:24.697123 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:24.697123 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:24.697123 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:24.697123 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:24.697123 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:24.697123 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:24.697123 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:24.697123 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:24.697123 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:24.697123 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:24.697123 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:24.698462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:24.698462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:24.698462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:24.698462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:24.698462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:24.698462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:24.698462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:24.698462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:24.698462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:24.698462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:24.698462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:24.698462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:24.698462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:24.698462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:24.698462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:24.698462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:24.698462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:24.699360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:24.699360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:24.699360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:24.699360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:24.699360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:24.699360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:24.699360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:24.699360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:24.703999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 377/753] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:24.705183 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35674e93-156f-4c8c-a762-0437fc95ae36 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:24.728831 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8906748-0048-4b65-8723-86e0ff624fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:24.729486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8906748-0048-4b65-8723-86e0ff624fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:24.729816 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8906748-0048-4b65-8723-86e0ff624fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:24.730127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8906748-0048-4b65-8723-86e0ff624fe3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:24.745654 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8906748-0048-4b65-8723-86e0ff624fe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:24.748440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8906748-0048-4b65-8723-86e0ff624fe3 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:24.767715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-e8906748-0048-4b65-8723-86e0ff624fe3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:24.767715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:24.767715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:24.767715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:24.767715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:24.767715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:24.767715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:24.767715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:24.767715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:24.767715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:24.767715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:24.767715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:24.767715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:24.767715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:24.767715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:24.768644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:24.768644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:24.768644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:24.768644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:24.768644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:24.768644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:24.768644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:24.768644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:24.768644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:24.768644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:24.768644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:24.768644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:24.768644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:24.768644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:24.768644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:24.768644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:24.768644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:24.770165 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:24.770165 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:24.770165 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:24.770165 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:24.770165 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 377/754] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:24.770165 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8906748-0048-4b65-8723-86e0ff624fe3 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:24.787459 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:24.821234 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35674e93-156f-4c8c-a762-0437fc95ae36 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:24.867561 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b614b8a-7974-4fdc-b08c-e3503fa27d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:24.868541 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b614b8a-7974-4fdc-b08c-e3503fa27d72 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:24.869135 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b614b8a-7974-4fdc-b08c-e3503fa27d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:24.869636 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b614b8a-7974-4fdc-b08c-e3503fa27d72 None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:24.870741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b614b8a-7974-4fdc-b08c-e3503fa27d72 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:24.871267 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b614b8a-7974-4fdc-b08c-e3503fa27d72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:24.872908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b614b8a-7974-4fdc-b08c-e3503fa27d72 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:24.891186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 378/755] 10.222.0.46 () {56 vars in 1133 bytes} [Tue Aug 2 08:47:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:24.892508 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b614b8a-7974-4fdc-b08c-e3503fa27d72 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:24.917570 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8906748-0048-4b65-8723-86e0ff624fe3 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:24.971667 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b4e40-8301-445a-8e9d-ac62cf485d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:24.972324 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b4e40-8301-445a-8e9d-ac62cf485d8d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:24.972605 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b4e40-8301-445a-8e9d-ac62cf485d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:24.972881 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22b4e40-8301-445a-8e9d-ac62cf485d8d None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:24.973632 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22b4e40-8301-445a-8e9d-ac62cf485d8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:24.974012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22b4e40-8301-445a-8e9d-ac62cf485d8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:24.975546 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22b4e40-8301-445a-8e9d-ac62cf485d8d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:24.994041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 378/756] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:24.995193 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b4e40-8301-445a-8e9d-ac62cf485d8d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:25.010725 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:25.018187 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b614b8a-7974-4fdc-b08c-e3503fa27d72 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:25.049536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:25.058105 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75349681-877d-4de9-b147-348f36c7a632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:25.058737 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75349681-877d-4de9-b147-348f36c7a632 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:25.059024 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75349681-877d-4de9-b147-348f36c7a632 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:25.059297 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75349681-877d-4de9-b147-348f36c7a632 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:25.059994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75349681-877d-4de9-b147-348f36c7a632 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:25.060299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75349681-877d-4de9-b147-348f36c7a632 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:25.061672 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75349681-877d-4de9-b147-348f36c7a632 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:25.078101 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 379/757] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:25.079097 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75349681-877d-4de9-b147-348f36c7a632 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:25.109168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22b4e40-8301-445a-8e9d-ac62cf485d8d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:25.143598 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8713c2-8701-4a2f-ba9d-7a2f427a0ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:25.144233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8713c2-8701-4a2f-ba9d-7a2f427a0ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:25.144515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8713c2-8701-4a2f-ba9d-7a2f427a0ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:25.144790 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8713c2-8701-4a2f-ba9d-7a2f427a0ba6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:25.156159 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8713c2-8701-4a2f-ba9d-7a2f427a0ba6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:25.157601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8713c2-8701-4a2f-ba9d-7a2f427a0ba6 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:25.174425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-6e8713c2-8701-4a2f-ba9d-7a2f427a0ba6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:25.174425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:25.174425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:25.174425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:25.174425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:25.174425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:25.174425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:25.174425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:25.174425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:25.174425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:25.174425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:25.174425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:25.174425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:25.174425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:25.174425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:25.175499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:25.175499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:25.175499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:25.175499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:25.175499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:25.175499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:25.175499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:25.175499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:25.175499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:25.175499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:25.175499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:25.175499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:25.175499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:25.175499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:25.175499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:25.175499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:25.175499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:25.176595 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:25.176595 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:25.176595 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:25.176595 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:25.176595 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 379/758] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:25.177094 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d44790-dbad-46a8-94e4-602af6321d07 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:25.177375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d44790-dbad-46a8-94e4-602af6321d07 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:25.178246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d44790-dbad-46a8-94e4-602af6321d07 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:25.182057 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 380/759] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:25.182057 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d44790-dbad-46a8-94e4-602af6321d07 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:25.194103 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75349681-877d-4de9-b147-348f36c7a632 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:25.228773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af21b3-892d-4e90-9b08-3026ebfd9a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:25.229394 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af21b3-892d-4e90-9b08-3026ebfd9a57 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:25.229679 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af21b3-892d-4e90-9b08-3026ebfd9a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:25.229984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af21b3-892d-4e90-9b08-3026ebfd9a57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:25.230776 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af21b3-892d-4e90-9b08-3026ebfd9a57 None admin] RBAC: Adding query filter params (role.id=8ce0850e15804f3d811e88cc9841e673, scope.project.id=4227a5b804c0414c99d407bb1415cf49, user.id=ff92ef3b9da74ccab3fbc43f852e3ad0) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:25.231087 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af21b3-892d-4e90-9b08-3026ebfd9a57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:25.232509 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7af21b3-892d-4e90-9b08-3026ebfd9a57 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:25.245916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 380/760] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:47:25 2022] GET /identity/v3/role_assignments?user.id=ff92ef3b9da74ccab3fbc43f852e3ad0&scope.project.id=4227a5b804c0414c99d407bb1415cf49&role.id=8ce0850e15804f3d811e88cc9841e673 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:25.245916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af21b3-892d-4e90-9b08-3026ebfd9a57 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:25.259256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:25.310037 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:25.321820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d44790-dbad-46a8-94e4-602af6321d07 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:25.363159 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093d504-f037-43b7-8cd8-67eae3682c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:25.363824 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093d504-f037-43b7-8cd8-67eae3682c3e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:25.364156 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093d504-f037-43b7-8cd8-67eae3682c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:25.364439 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093d504-f037-43b7-8cd8-67eae3682c3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:25.372879 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093d504-f037-43b7-8cd8-67eae3682c3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:25.374284 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093d504-f037-43b7-8cd8-67eae3682c3e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:25.384016 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af21b3-892d-4e90-9b08-3026ebfd9a57 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:25.389950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-4093d504-f037-43b7-8cd8-67eae3682c3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:25.389950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:25.389950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:25.389950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:25.389950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:25.389950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:25.389950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:25.389950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:25.389950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:25.389950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:25.389950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:25.389950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:25.389950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:25.389950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:25.389950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:25.390904 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:25.390904 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:25.390904 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:25.390904 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:25.390904 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:25.390904 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:25.390904 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:25.390904 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:25.390904 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:25.390904 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:25.390904 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:25.390904 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:25.390904 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:25.390904 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:25.390904 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:25.390904 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:25.390904 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:25.391805 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:25.391805 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:25.391805 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:25.391805 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:25.391805 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:25.391805 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:25.391805 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:25.391805 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:25.391805 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 381/761] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:25.392738 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa29d9-60cc-446c-8f5c-a7e5573046fc None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:25.392948 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa29d9-60cc-446c-8f5c-a7e5573046fc None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:25.393222 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa29d9-60cc-446c-8f5c-a7e5573046fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:25.413245 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-23fa29d9-60cc-446c-8f5c-a7e5573046fc None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:25.414480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-23fa29d9-60cc-446c-8f5c-a7e5573046fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:25.433866 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a41bc-07e8-4bec-961f-215e86466a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:25.434536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a41bc-07e8-4bec-961f-215e86466a08 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:25.434817 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a41bc-07e8-4bec-961f-215e86466a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:25.435091 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a41bc-07e8-4bec-961f-215e86466a08 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:25.436310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a41bc-07e8-4bec-961f-215e86466a08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:25.436618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a41bc-07e8-4bec-961f-215e86466a08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:25.438548 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5a41bc-07e8-4bec-961f-215e86466a08 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:25.454922 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 381/762] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:25.455695 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a41bc-07e8-4bec-961f-215e86466a08 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:25.494002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 382/763] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:47:25.494923 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa29d9-60cc-446c-8f5c-a7e5573046fc None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:25.569691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:25.577575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a41bc-07e8-4bec-961f-215e86466a08 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:25.594732 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:25.621351 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa29d9-60cc-446c-8f5c-a7e5573046fc None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:25.622774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6a982-8189-4507-a6bc-a760599f6302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:25.623712 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6a982-8189-4507-a6bc-a760599f6302 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:25.624926 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6a982-8189-4507-a6bc-a760599f6302 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:25.625352 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6a982-8189-4507-a6bc-a760599f6302 None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:25.626270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6a982-8189-4507-a6bc-a760599f6302 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:25.626721 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6a982-8189-4507-a6bc-a760599f6302 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:25.628301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6a982-8189-4507-a6bc-a760599f6302 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:25.643105 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 382/764] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:25.644811 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6a982-8189-4507-a6bc-a760599f6302 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:25.659426 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2c15b-7781-46c6-85c6-cdb306d48d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:25.660225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2c15b-7781-46c6-85c6-cdb306d48d77 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:25.660604 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2c15b-7781-46c6-85c6-cdb306d48d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:25.661153 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2c15b-7781-46c6-85c6-cdb306d48d77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:25.670721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2c15b-7781-46c6-85c6-cdb306d48d77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:25.672819 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2c15b-7781-46c6-85c6-cdb306d48d77 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:25.689016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-77a2c15b-7781-46c6-85c6-cdb306d48d77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:25.689016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:25.689016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:25.689016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:25.689016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:25.689016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:25.689016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:25.689016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:25.689016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:25.689016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:25.689016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:25.689016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:25.689016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:25.689016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:25.689016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:25.690201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:25.690201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:25.690201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:25.690201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:25.690201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:25.690201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:25.690201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:25.690201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:25.690201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:25.690201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:25.690201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:25.690201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:25.690201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:25.690201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:25.690201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:25.690201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:25.690201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:25.691310 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:25.691310 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:25.691310 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:25.691310 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:25.691310 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:25.691310 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:25.691310 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:25.691310 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:25.692955 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 383/765] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:25.693918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2c15b-7781-46c6-85c6-cdb306d48d77 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:25.766753 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6a982-8189-4507-a6bc-a760599f6302 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:25.804037 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455b498-3bd6-413a-9f67-8228dfff9087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:25.804772 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455b498-3bd6-413a-9f67-8228dfff9087 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:25.805128 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455b498-3bd6-413a-9f67-8228dfff9087 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:25.805467 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7455b498-3bd6-413a-9f67-8228dfff9087 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:25.806262 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455b498-3bd6-413a-9f67-8228dfff9087 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:25.807806 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7455b498-3bd6-413a-9f67-8228dfff9087 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:25.812722 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2c15b-7781-46c6-85c6-cdb306d48d77 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:25.817678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-7455b498-3bd6-413a-9f67-8228dfff9087 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 08:47:25.817678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:25.817678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:47:25.817678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:47:25.817678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:47:25.817678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:47:25.817678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:47:25.817678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:25.817678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 08:47:25.817678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:25.817678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:25.817678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:25.817678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:25.817678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:25.817678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:25.817678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:25.825531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:25.825531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:25.825531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:25.825531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:25.825531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:25.825531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:25.825531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:25.825531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:47:25.825531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:47:25.825531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:47:25.825531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:47:25.825531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:25.825531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:25.825531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:25.825531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:25.825531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:25.826445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:25.826445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:25.826445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:25.826445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:25.826445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:25.826445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:25.826445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:25.826445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:25.826445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:25.826445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:25.826445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:25.826445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:47:25.826445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:25.826445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 08:47:25.826445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:25.826445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 383/766] 10.222.0.46 () {56 vars in 1129 bytes} [Tue Aug 2 08:47:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:25.827303 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455b498-3bd6-413a-9f67-8228dfff9087 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:25.846306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:25.854682 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8b577-47c8-4f58-af7c-6fdfbd93e8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:25.855535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8b577-47c8-4f58-af7c-6fdfbd93e8c5 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:25.855997 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8b577-47c8-4f58-af7c-6fdfbd93e8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:25.856465 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e8b577-47c8-4f58-af7c-6fdfbd93e8c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:25.857428 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e8b577-47c8-4f58-af7c-6fdfbd93e8c5 None admin] RBAC: Adding query filter params (role.id=391becda2bf04438b8d1763ab99acf1b, scope.project.id=4227a5b804c0414c99d407bb1415cf49, user.id=ed8114b7a28741268fdff71c866dbc77) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:25.858008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e8b577-47c8-4f58-af7c-6fdfbd93e8c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:25.859622 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e8b577-47c8-4f58-af7c-6fdfbd93e8c5 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:25.863753 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:25.866644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 384/767] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:47:25 2022] GET /identity/v3/role_assignments?user.id=ed8114b7a28741268fdff71c866dbc77&scope.project.id=4227a5b804c0414c99d407bb1415cf49&role.id=391becda2bf04438b8d1763ab99acf1b => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:25.867672 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8b577-47c8-4f58-af7c-6fdfbd93e8c5 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:25.946122 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7455b498-3bd6-413a-9f67-8228dfff9087 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:25.983971 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e8b577-47c8-4f58-af7c-6fdfbd93e8c5 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:25.986437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd0db5-0f17-4484-9696-f26a9ddceff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:25.987069 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd0db5-0f17-4484-9696-f26a9ddceff8 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:25.988152 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd0db5-0f17-4484-9696-f26a9ddceff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:25.988152 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd0db5-0f17-4484-9696-f26a9ddceff8 None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:25.988391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd0db5-0f17-4484-9696-f26a9ddceff8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:25.988676 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd0db5-0f17-4484-9696-f26a9ddceff8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:25.998434 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd0db5-0f17-4484-9696-f26a9ddceff8 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:26.018639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 384/768] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:26.024599 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cd4cd-3290-46aa-9d6c-02692c016f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:26.025311 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd0db5-0f17-4484-9696-f26a9ddceff8 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:26.025595 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cd4cd-3290-46aa-9d6c-02692c016f9f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:26.026067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cd4cd-3290-46aa-9d6c-02692c016f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:26.026520 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cd4cd-3290-46aa-9d6c-02692c016f9f None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:26.027368 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6cd4cd-3290-46aa-9d6c-02692c016f9f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:26.027822 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6cd4cd-3290-46aa-9d6c-02692c016f9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:26.029340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6cd4cd-3290-46aa-9d6c-02692c016f9f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:26.034835 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 385/769] 10.222.0.46 () {56 vars in 1124 bytes} [Tue Aug 2 08:47:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:26.040971 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cd4cd-3290-46aa-9d6c-02692c016f9f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:26.094649 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:26.150191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:26.156118 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd0db5-0f17-4484-9696-f26a9ddceff8 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:26.167622 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cd4cd-3290-46aa-9d6c-02692c016f9f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:26.198485 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190d2da-3063-4292-bba8-787b377c8731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:26.199153 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190d2da-3063-4292-bba8-787b377c8731 None admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:26.199431 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190d2da-3063-4292-bba8-787b377c8731 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:26.199703 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190d2da-3063-4292-bba8-787b377c8731 None admin] PATH_INFO: `/v3/projects/4227a5b804c0414c99d407bb1415cf49/users/3191d804e8dc4b1a85f490319833ec07/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:26.203030 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190d2da-3063-4292-bba8-787b377c8731 None admin] RBAC: Authorizing `identity:create_grant(project_id=4227a5b804c0414c99d407bb1415cf49, user_id=3191d804e8dc4b1a85f490319833ec07, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:26.205585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190d2da-3063-4292-bba8-787b377c8731 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:26.213088 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1917b-b1bd-40b5-a1cd-ecee4ee9b79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:26.213704 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1917b-b1bd-40b5-a1cd-ecee4ee9b79a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:26.214081 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1917b-b1bd-40b5-a1cd-ecee4ee9b79a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:26.214359 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1917b-b1bd-40b5-a1cd-ecee4ee9b79a None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:26.215071 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1917b-b1bd-40b5-a1cd-ecee4ee9b79a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:26.215344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1917b-b1bd-40b5-a1cd-ecee4ee9b79a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:26.216682 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1917b-b1bd-40b5-a1cd-ecee4ee9b79a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:26.225992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 386/770] 10.222.0.46 () {56 vars in 1124 bytes} [Tue Aug 2 08:47:26 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:26.225992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20704d32-9573-4a3e-b567-b7dc4d865c42 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:26.225992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20704d32-9573-4a3e-b567-b7dc4d865c42 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:26.225992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20704d32-9573-4a3e-b567-b7dc4d865c42 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:26.228870 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 387/771] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:26.229652 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20704d32-9573-4a3e-b567-b7dc4d865c42 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:26.293110 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 385/772] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:47:26 2022] PUT /identity/v3/projects/4227a5b804c0414c99d407bb1415cf49/users/3191d804e8dc4b1a85f490319833ec07/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:47:26.295754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d699382-36c0-4b34-8db8-98100d8ab608 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:26.296288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d699382-36c0-4b34-8db8-98100d8ab608 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:26.296835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d699382-36c0-4b34-8db8-98100d8ab608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:26.323447 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-6d699382-36c0-4b34-8db8-98100d8ab608 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:26.325894 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-6d699382-36c0-4b34-8db8-98100d8ab608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:26.339589 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20704d32-9573-4a3e-b567-b7dc4d865c42 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:26.343162 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:26.378900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e515f40-d077-4edd-903a-ede67fef290b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:26.379638 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e515f40-d077-4edd-903a-ede67fef290b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:26.379919 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e515f40-d077-4edd-903a-ede67fef290b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:26.380195 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e515f40-d077-4edd-903a-ede67fef290b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:26.381557 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e515f40-d077-4edd-903a-ede67fef290b None admin] RBAC: Adding query filter params (interface=public, service_id=ed3a0e7aa667486e91dc6d96e06041b4, region_id=RegionOne) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:26.381876 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e515f40-d077-4edd-903a-ede67fef290b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:26.383928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e515f40-d077-4edd-903a-ede67fef290b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:26.391841 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 388/773] 10.222.0.46 () {56 vars in 1251 bytes} [Tue Aug 2 08:47:26 2022] GET /identity/v3/endpoints?service_id=ed3a0e7aa667486e91dc6d96e06041b4&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:26.400788 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e515f40-d077-4edd-903a-ede67fef290b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:26.469994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 386/774] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:47:26.513329 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d699382-36c0-4b34-8db8-98100d8ab608 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:26.546294 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e515f40-d077-4edd-903a-ede67fef290b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:26.592550 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62523b69-6a96-41de-987d-b7778ed22b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:26.593720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62523b69-6a96-41de-987d-b7778ed22b5f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:26.595119 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62523b69-6a96-41de-987d-b7778ed22b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:26.595606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62523b69-6a96-41de-987d-b7778ed22b5f None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:26.596464 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62523b69-6a96-41de-987d-b7778ed22b5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:26.598078 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62523b69-6a96-41de-987d-b7778ed22b5f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:26.606225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 389/775] 10.222.0.46 () {56 vars in 1087 bytes} [Tue Aug 2 08:47:26 2022] GET /identity/v3/services => generated 772 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:26.619995 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:26.638841 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d699382-36c0-4b34-8db8-98100d8ab608 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:26.699989 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e62e0c-7ae6-4825-a32b-11cec246d1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:26.700657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e62e0c-7ae6-4825-a32b-11cec246d1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:26.700940 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e62e0c-7ae6-4825-a32b-11cec246d1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:26.701284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e62e0c-7ae6-4825-a32b-11cec246d1a8 None admin] PATH_INFO: `/v3/services/network` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:26.701952 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e62e0c-7ae6-4825-a32b-11cec246d1a8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:26.703359 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e62e0c-7ae6-4825-a32b-11cec246d1a8 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:26.719761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-a0e62e0c-7ae6-4825-a32b-11cec246d1a8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 08:47:26.719761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:26.719761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:47:26.719761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:47:26.719761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:47:26.719761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:47:26.719761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:47:26.719761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:26.719761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 08:47:26.719761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:26.719761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:26.719761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:26.719761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:26.719761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:26.719761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:26.719761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:26.719761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:26.720848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:26.720848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:26.720848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:26.720848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:26.720848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:26.720848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:26.720848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:47:26.720848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:47:26.720848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:47:26.720848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:47:26.720848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:26.720848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:26.720848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:26.720848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:26.720848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:26.720848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:26.721682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:26.721682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:26.721682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:26.721682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:26.721682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:26.721682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:26.721682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:26.721682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:26.721682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:26.721682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:26.721682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:47:26.721682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:26.721682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 08:47:26.721682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:26.721682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 387/776] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:26 2022] GET /identity/v3/services/network => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:26.728095 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62523b69-6a96-41de-987d-b7778ed22b5f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:26.775616 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:26.857514 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62523b69-6a96-41de-987d-b7778ed22b5f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:26.898750 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cd90b-251b-4cf6-8fbf-11e54d0937f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:26.901559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cd90b-251b-4cf6-8fbf-11e54d0937f4 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:26.901559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cd90b-251b-4cf6-8fbf-11e54d0937f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:26.901559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3cd90b-251b-4cf6-8fbf-11e54d0937f4 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:26.901559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3cd90b-251b-4cf6-8fbf-11e54d0937f4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:26.901559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3cd90b-251b-4cf6-8fbf-11e54d0937f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:26.902711 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3cd90b-251b-4cf6-8fbf-11e54d0937f4 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:26.907641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 390/777] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 2 08:47:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:26.913539 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e62e0c-7ae6-4825-a32b-11cec246d1a8 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:27.087771 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e62e0c-7ae6-4825-a32b-11cec246d1a8 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:27.142810 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bcf223-7638-4439-bde2-fd91bce79467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:27.143345 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bcf223-7638-4439-bde2-fd91bce79467 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:27.143624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bcf223-7638-4439-bde2-fd91bce79467 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:27.143894 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bcf223-7638-4439-bde2-fd91bce79467 None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:27.144573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bcf223-7638-4439-bde2-fd91bce79467 None admin] RBAC: Adding query filter params (type=network) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:27.144873 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bcf223-7638-4439-bde2-fd91bce79467 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:27.146801 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bcf223-7638-4439-bde2-fd91bce79467 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:27.154702 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 388/778] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 2 08:47:26 2022] GET /identity/v3/services?type=network => generated 350 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:27.160086 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cd90b-251b-4cf6-8fbf-11e54d0937f4 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:27.278246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:27.284572 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3cd90b-251b-4cf6-8fbf-11e54d0937f4 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:27.318496 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04cad30-c858-4eb0-a8e5-84b581d98c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:27.319134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cad30-c858-4eb0-a8e5-84b581d98c1a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:27.319416 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cad30-c858-4eb0-a8e5-84b581d98c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:27.319692 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04cad30-c858-4eb0-a8e5-84b581d98c1a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:27.320545 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04cad30-c858-4eb0-a8e5-84b581d98c1a None admin] RBAC: Adding query filter params (interface=public, service_id=5e0228c9f01f4b15a3d8078982ca9dc0, region_id=RegionOne) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:27.321128 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04cad30-c858-4eb0-a8e5-84b581d98c1a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:27.322771 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04cad30-c858-4eb0-a8e5-84b581d98c1a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:27.329779 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 391/779] 10.222.0.46 () {56 vars in 1251 bytes} [Tue Aug 2 08:47:27 2022] GET /identity/v3/endpoints?service_id=5e0228c9f01f4b15a3d8078982ca9dc0&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:27.335497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bcf223-7638-4439-bde2-fd91bce79467 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:27.368579 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:27.509587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bcf223-7638-4439-bde2-fd91bce79467 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:27.550031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e519c-bead-495e-a662-6d84b4bda7a1 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:27.550967 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e519c-bead-495e-a662-6d84b4bda7a1 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:27.551452 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e519c-bead-495e-a662-6d84b4bda7a1 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:27.553944 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 392/780] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:27.558211 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d9889-a47d-4f17-89aa-30b94c22fc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:27.558827 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d9889-a47d-4f17-89aa-30b94c22fc2c None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:27.559107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d9889-a47d-4f17-89aa-30b94c22fc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:27.559378 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d9889-a47d-4f17-89aa-30b94c22fc2c None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:27.559823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b92b0-98b8-43fc-925e-c030f56aa1a6 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:27.560021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d9889-a47d-4f17-89aa-30b94c22fc2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:27.560492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b92b0-98b8-43fc-925e-c030f56aa1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:27.560980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b92b0-98b8-43fc-925e-c030f56aa1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:27.561385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d9889-a47d-4f17-89aa-30b94c22fc2c None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:27.566766 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 389/781] 10.222.0.46 () {56 vars in 1087 bytes} [Tue Aug 2 08:47:27 2022] GET /identity/v3/services => generated 772 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:27.582235 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-3c1b92b0-98b8-43fc-925e-c030f56aa1a6 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:27.584177 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3c1b92b0-98b8-43fc-925e-c030f56aa1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:27.661629 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 393/782] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:47:27.685698 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d9889-a47d-4f17-89aa-30b94c22fc2c None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:27.775371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:27.796095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d9889-a47d-4f17-89aa-30b94c22fc2c None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:27.833241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07318b25-0d5f-4d62-b6bd-57e139ef706e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:27.833991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07318b25-0d5f-4d62-b6bd-57e139ef706e None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:27.834280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07318b25-0d5f-4d62-b6bd-57e139ef706e None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:27.834560 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07318b25-0d5f-4d62-b6bd-57e139ef706e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:27.848104 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07318b25-0d5f-4d62-b6bd-57e139ef706e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:47:27.854384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-07318b25-0d5f-4d62-b6bd-57e139ef706e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:47:27.854384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:27.854384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:27.854384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:27.854384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:27.854384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:27.854384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:27.854384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:27.854384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:27.854384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:27.854384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:27.854384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:27.854384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:47:27.854384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:47:27.854384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:47:27.854384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:47:27.855403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:47:27.855403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:47:27.855403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:47:27.855403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:47:27.855403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:27.855403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:27.855403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:27.855403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:27.855403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:27.855403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:27.855403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:27.855403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:27.855403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:27.855403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:27.855403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:27.855403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:27.856254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:27.856254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:27.856254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:27.856254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:27.856254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:47:27.856254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:47:27.856254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:47:27.856254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:47:27.856254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:47:27.856254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:47:27.856254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:47:27.856254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:27.856254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 390/783] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:47:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:27.862091 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b92b0-98b8-43fc-925e-c030f56aa1a6 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:27.976778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:27.992583 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b92b0-98b8-43fc-925e-c030f56aa1a6 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:28.044373 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558caf57-ea69-4794-bc78-e493a14ebf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:28.044917 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558caf57-ea69-4794-bc78-e493a14ebf3b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:28.045193 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558caf57-ea69-4794-bc78-e493a14ebf3b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:28.045468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558caf57-ea69-4794-bc78-e493a14ebf3b None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:28.046210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558caf57-ea69-4794-bc78-e493a14ebf3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:28.046511 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558caf57-ea69-4794-bc78-e493a14ebf3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:28.047873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558caf57-ea69-4794-bc78-e493a14ebf3b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:28.061643 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 394/784] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:47:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:28.066849 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07318b25-0d5f-4d62-b6bd-57e139ef706e None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:28.197499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07318b25-0d5f-4d62-b6bd-57e139ef706e None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:28.235983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1220de-052e-41c9-af93-63884a845239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:28.236649 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1220de-052e-41c9-af93-63884a845239 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:28.236931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1220de-052e-41c9-af93-63884a845239 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:28.237210 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1220de-052e-41c9-af93-63884a845239 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:28.245197 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:28.250653 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1220de-052e-41c9-af93-63884a845239 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:28.252032 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1220de-052e-41c9-af93-63884a845239 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:28.268720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-bc1220de-052e-41c9-af93-63884a845239 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:28.268720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:28.268720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:28.268720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:28.268720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:28.268720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:28.268720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:28.268720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:28.268720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:28.268720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:28.268720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:28.268720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:28.268720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:28.268720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:28.268720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:28.269665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:28.269665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:28.269665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:28.269665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:28.269665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:28.269665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:28.269665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:28.269665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:28.269665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:28.269665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:28.269665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:28.269665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:28.269665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:28.269665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:28.269665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:28.269665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:28.269665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:28.270646 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:28.270646 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:28.270646 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:28.270646 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:28.270646 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 391/785] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:28.276514 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558caf57-ea69-4794-bc78-e493a14ebf3b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:28.381809 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558caf57-ea69-4794-bc78-e493a14ebf3b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:28.399099 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7fc55-20d5-468f-86d1-6841cb2b586e None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:28.399729 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7fc55-20d5-468f-86d1-6841cb2b586e None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:28.400102 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7fc55-20d5-468f-86d1-6841cb2b586e None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:28.401778 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 392/786] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:47:28.417758 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287fcbe-9ca7-4d07-9344-409ce2474bd1 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:28.418063 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287fcbe-9ca7-4d07-9344-409ce2474bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:28.418351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4287fcbe-9ca7-4d07-9344-409ce2474bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:28.428630 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964ffdb-e324-496d-8be9-5c0fdfac3df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:28.429259 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964ffdb-e324-496d-8be9-5c0fdfac3df4 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:28.429542 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964ffdb-e324-496d-8be9-5c0fdfac3df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:28.429815 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964ffdb-e324-496d-8be9-5c0fdfac3df4 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:28.430544 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f964ffdb-e324-496d-8be9-5c0fdfac3df4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:28.430854 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f964ffdb-e324-496d-8be9-5c0fdfac3df4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:28.432220 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f964ffdb-e324-496d-8be9-5c0fdfac3df4 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:28.444251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-4287fcbe-9ca7-4d07-9344-409ce2474bd1 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:28.445552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4287fcbe-9ca7-4d07-9344-409ce2474bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:28.459881 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 395/787] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:28.466441 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964ffdb-e324-496d-8be9-5c0fdfac3df4 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:28.488761 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:28.536338 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 393/788] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:47:28.562327 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287fcbe-9ca7-4d07-9344-409ce2474bd1 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:28.645567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f964ffdb-e324-496d-8be9-5c0fdfac3df4 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:28.713783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4287fcbe-9ca7-4d07-9344-409ce2474bd1 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:28.728869 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2932d-b3c8-404d-b776-7c32d011c490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:28.730447 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2932d-b3c8-404d-b776-7c32d011c490 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:28.730929 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2932d-b3c8-404d-b776-7c32d011c490 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:28.732086 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2932d-b3c8-404d-b776-7c32d011c490 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:28.743003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2932d-b3c8-404d-b776-7c32d011c490 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:28.744782 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2932d-b3c8-404d-b776-7c32d011c490 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:28.750251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:28.758357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-89f2932d-b3c8-404d-b776-7c32d011c490 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 08:47:28.758357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:28.758357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:28.758357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:28.758357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:28.758357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:28.758357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:28.758357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:28.758357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:28.758357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:28.758357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:28.758357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:28.758357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:28.758357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:28.758357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:28.758357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:28.769801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:28.769801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:28.769801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:28.769801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:28.769801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:28.769801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:28.769801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:28.769801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:28.769801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:28.769801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:28.769801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:28.769801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:28.769801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:28.769801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:28.769801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:28.769801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:28.769801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:28.769801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:28.770824 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:28.770824 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:28.770824 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:28.770824 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:28.770824 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:28.770824 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:28.770824 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:28.770824 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:28.770824 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 08:47:28.770824 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:28.770824 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 396/789] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:47:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:28.770824 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2932d-b3c8-404d-b776-7c32d011c490 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:28.793831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7c0c4-ef51-416d-b125-845c19d35049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:28.794536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7c0c4-ef51-416d-b125-845c19d35049 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:28.795140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7c0c4-ef51-416d-b125-845c19d35049 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:28.795140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7c0c4-ef51-416d-b125-845c19d35049 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:28.834353 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f7c0c4-ef51-416d-b125-845c19d35049 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:47:28.839070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-43f7c0c4-ef51-416d-b125-845c19d35049 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:28.839070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:28.839070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:28.839070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:28.839070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:28.839070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:28.839070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:28.839070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:28.839070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:28.839070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:28.839070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:28.839070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:28.839070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:47:28.839070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:47:28.839070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:47:28.839070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:47:28.840017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:47:28.840017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:47:28.840017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:47:28.840017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:47:28.840017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:28.840017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:28.840017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:28.840017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:28.840017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:28.840017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:28.840017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:28.840017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:28.840017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:28.840017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:28.840017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:28.840017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:28.841566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:28.841566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:28.841566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:28.841566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:28.841566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:47:28.841566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:47:28.841566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:47:28.841566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:47:28.841566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:47:28.841566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:47:28.841566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:28.841566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:28.841566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 394/790] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 2 08:47:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:28.846224 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7c0c4-ef51-416d-b125-845c19d35049 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:28.853853 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:28.888819 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2932d-b3c8-404d-b776-7c32d011c490 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:28.926242 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055331fb-4956-4db7-81a1-c89554496165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:28.927263 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055331fb-4956-4db7-81a1-c89554496165 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:28.928274 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055331fb-4956-4db7-81a1-c89554496165 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:28.928746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055331fb-4956-4db7-81a1-c89554496165 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:28.929910 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055331fb-4956-4db7-81a1-c89554496165 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:28.930422 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055331fb-4956-4db7-81a1-c89554496165 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:28.932080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055331fb-4956-4db7-81a1-c89554496165 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:28.955585 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 397/791] 10.222.0.46 () {56 vars in 1135 bytes} [Tue Aug 2 08:47:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:28.957639 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a46e8d-550c-4de8-8732-2e9a274a5790 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:28.958166 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a46e8d-550c-4de8-8732-2e9a274a5790 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:28.958971 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a46e8d-550c-4de8-8732-2e9a274a5790 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:28.961388 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 398/792] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:28.962771 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a46e8d-550c-4de8-8732-2e9a274a5790 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:29.032694 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7c0c4-ef51-416d-b125-845c19d35049 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:29.080359 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b81b2-518a-4225-8dbf-3b65e623db77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:29.080984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b81b2-518a-4225-8dbf-3b65e623db77 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:29.081273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b81b2-518a-4225-8dbf-3b65e623db77 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:29.081555 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b81b2-518a-4225-8dbf-3b65e623db77 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:29.082315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b81b2-518a-4225-8dbf-3b65e623db77 None admin] RBAC: Adding query filter params (name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:29.082618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b81b2-518a-4225-8dbf-3b65e623db77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:29.084121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b81b2-518a-4225-8dbf-3b65e623db77 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:29.084633 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a46e8d-550c-4de8-8732-2e9a274a5790 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:29.095039 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 395/793] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 2 08:47:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:29.096760 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da2b1f-efde-499b-b2a3-4ef66fc90061 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:29.097205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da2b1f-efde-499b-b2a3-4ef66fc90061 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:29.097622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da2b1f-efde-499b-b2a3-4ef66fc90061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:29.121280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-32da2b1f-efde-499b-b2a3-4ef66fc90061 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:29.122254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-32da2b1f-efde-499b-b2a3-4ef66fc90061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:29.127359 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011e48c-be9e-4d76-82db-a6a17eb3ce57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:29.128283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011e48c-be9e-4d76-82db-a6a17eb3ce57 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:29.128833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011e48c-be9e-4d76-82db-a6a17eb3ce57 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:29.129301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011e48c-be9e-4d76-82db-a6a17eb3ce57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:29.141271 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011e48c-be9e-4d76-82db-a6a17eb3ce57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:29.143224 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011e48c-be9e-4d76-82db-a6a17eb3ce57 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:29.151404 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:29.163635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-9011e48c-be9e-4d76-82db-a6a17eb3ce57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:29.163635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:29.163635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:29.163635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:29.163635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:29.163635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:29.163635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:29.163635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:29.163635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:29.163635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:29.163635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:29.163635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:29.163635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:29.163635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:29.163635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:29.164549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:29.164549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:29.164549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:29.164549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:29.164549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:29.164549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:29.164549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:29.164549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:29.164549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:29.164549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:29.164549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:29.164549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:29.164549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:29.164549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:29.164549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:29.164549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:29.164549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:29.165485 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:29.165485 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:29.165485 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:29.165485 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:29.167846 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 399/794] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:29.168851 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011e48c-be9e-4d76-82db-a6a17eb3ce57 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:29.169342 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:29.219833 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 396/795] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:47:29.221031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da2b1f-efde-499b-b2a3-4ef66fc90061 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:29.296108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011e48c-be9e-4d76-82db-a6a17eb3ce57 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:29.333121 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2c1f6-6679-4f16-b3a1-291f6a017d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:29.334374 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2c1f6-6679-4f16-b3a1-291f6a017d9a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:29.334636 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2c1f6-6679-4f16-b3a1-291f6a017d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:29.334910 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2c1f6-6679-4f16-b3a1-291f6a017d9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:29.345501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b2c1f6-6679-4f16-b3a1-291f6a017d9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:29.347036 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b2c1f6-6679-4f16-b3a1-291f6a017d9a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:29.348073 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da2b1f-efde-499b-b2a3-4ef66fc90061 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:29.360756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-20b2c1f6-6679-4f16-b3a1-291f6a017d9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:29.360756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:29.360756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:29.360756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:29.360756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:29.360756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:29.360756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:29.360756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:29.360756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:29.360756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:29.360756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:29.360756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:29.360756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:29.360756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:29.360756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:29.361651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:29.361651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:29.361651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:29.361651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:29.361651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:29.361651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:29.361651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:29.361651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:29.361651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:29.361651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:29.361651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:29.361651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:29.361651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:29.361651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:29.361651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:29.361651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:29.361651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:29.362606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:29.362606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:29.362606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:29.362606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:29.362606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 400/796] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:29.362881 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2c1f6-6679-4f16-b3a1-291f6a017d9a None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:29.390528 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b6085-8f37-40ec-ad09-ed656db4c04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:29.391133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b6085-8f37-40ec-ad09-ed656db4c04e None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:29.391414 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b6085-8f37-40ec-ad09-ed656db4c04e None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:29.391761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b6085-8f37-40ec-ad09-ed656db4c04e None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:29.392352 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b6085-8f37-40ec-ad09-ed656db4c04e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:29.392650 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b6085-8f37-40ec-ad09-ed656db4c04e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:29.394017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b6085-8f37-40ec-ad09-ed656db4c04e None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:29.401652 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:29.409921 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 397/797] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:29.409921 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f5c4c-3a40-405a-b43d-887383b902e7 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:29.409921 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f5c4c-3a40-405a-b43d-887383b902e7 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:29.409921 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882f5c4c-3a40-405a-b43d-887383b902e7 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:29.409921 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 398/798] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:29.409921 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f5c4c-3a40-405a-b43d-887383b902e7 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:29.440411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:29.476725 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b2c1f6-6679-4f16-b3a1-291f6a017d9a None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:29.509809 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e013be-9944-4a27-a686-ec7557781843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:29.510482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e013be-9944-4a27-a686-ec7557781843 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:29.510770 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e013be-9944-4a27-a686-ec7557781843 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:29.511047 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e013be-9944-4a27-a686-ec7557781843 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:29.512651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882f5c4c-3a40-405a-b43d-887383b902e7 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:29.522467 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e013be-9944-4a27-a686-ec7557781843 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:47:29.527906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-74e013be-9944-4a27-a686-ec7557781843 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:47:29.527906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:29.527906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:29.527906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:29.527906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:29.527906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:29.527906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:29.527906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:29.527906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:29.527906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:29.527906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:29.527906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:29.527906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:47:29.527906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:47:29.527906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:47:29.527906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:47:29.528855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:47:29.528855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:47:29.528855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:47:29.528855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:47:29.528855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:29.528855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:29.528855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:29.528855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:29.528855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:29.528855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:29.528855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:29.528855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:29.528855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:29.528855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:29.528855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:29.528855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:29.529710 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:29.529710 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:29.529710 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:29.529710 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:29.529710 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:47:29.529710 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:47:29.529710 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:47:29.529710 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:47:29.529710 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:47:29.529710 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:47:29.529710 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:47:29.529710 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:29.529710 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 401/799] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:47:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:29.530805 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e013be-9944-4a27-a686-ec7557781843 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:29.552214 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef43f7-1330-4d29-a45c-5a415d3de672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:29.552950 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef43f7-1330-4d29-a45c-5a415d3de672 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:29.553316 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef43f7-1330-4d29-a45c-5a415d3de672 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:29.553661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef43f7-1330-4d29-a45c-5a415d3de672 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:29.554388 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef43f7-1330-4d29-a45c-5a415d3de672 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:29.554692 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef43f7-1330-4d29-a45c-5a415d3de672 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:29.556048 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef43f7-1330-4d29-a45c-5a415d3de672 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:29.567861 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 399/800] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:29.568910 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139afac4-fd12-4860-8ef5-1733a8b48118 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:29.569198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139afac4-fd12-4860-8ef5-1733a8b48118 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:29.569463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139afac4-fd12-4860-8ef5-1733a8b48118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:29.587299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-139afac4-fd12-4860-8ef5-1733a8b48118 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:29.588466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-139afac4-fd12-4860-8ef5-1733a8b48118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:29.642501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:29.651534 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e013be-9944-4a27-a686-ec7557781843 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:29.670101 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 400/801] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:47:29.670101 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139afac4-fd12-4860-8ef5-1733a8b48118 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:29.684164 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:29.705238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2dc5a-5a6d-4361-9c96-170f9cbfde8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:29.705878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2dc5a-5a6d-4361-9c96-170f9cbfde8e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:29.706183 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2dc5a-5a6d-4361-9c96-170f9cbfde8e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:29.706456 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2dc5a-5a6d-4361-9c96-170f9cbfde8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:29.713845 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2dc5a-5a6d-4361-9c96-170f9cbfde8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:29.715330 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2dc5a-5a6d-4361-9c96-170f9cbfde8e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:29.727291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-16d2dc5a-5a6d-4361-9c96-170f9cbfde8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:29.727291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:29.727291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:29.727291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:29.727291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:29.727291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:29.727291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:29.727291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:29.727291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:29.727291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:29.727291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:29.727291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:29.727291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:29.727291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:29.727291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:29.728185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:29.728185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:29.728185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:29.728185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:29.728185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:29.728185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:29.728185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:29.728185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:29.728185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:29.728185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:29.728185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:29.728185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:29.728185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:29.728185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:29.728185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:29.728185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:29.728185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:29.729104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:29.729104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:29.729104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:29.729104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:29.729104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:29.729104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:29.729104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:29.729104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:29.729104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 402/802] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:29.729104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2dc5a-5a6d-4361-9c96-170f9cbfde8e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:29.774272 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139afac4-fd12-4860-8ef5-1733a8b48118 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:29.811284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15d441-1fe4-49c2-a33e-24d6fca20e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:29.812007 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15d441-1fe4-49c2-a33e-24d6fca20e7e None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:29.812295 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15d441-1fe4-49c2-a33e-24d6fca20e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:29.812581 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15d441-1fe4-49c2-a33e-24d6fca20e7e None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:29.813293 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15d441-1fe4-49c2-a33e-24d6fca20e7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:29.813606 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15d441-1fe4-49c2-a33e-24d6fca20e7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:29.815259 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15d441-1fe4-49c2-a33e-24d6fca20e7e None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:29.825194 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 401/803] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:47:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:29.826081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15d441-1fe4-49c2-a33e-24d6fca20e7e None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:29.839358 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2dc5a-5a6d-4361-9c96-170f9cbfde8e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:29.873939 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b888d9-5194-4de8-adbe-aace797c6bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:29.874581 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b888d9-5194-4de8-adbe-aace797c6bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:29.874862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b888d9-5194-4de8-adbe-aace797c6bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:29.875136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b888d9-5194-4de8-adbe-aace797c6bb8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:29.882372 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b888d9-5194-4de8-adbe-aace797c6bb8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:29.883803 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b888d9-5194-4de8-adbe-aace797c6bb8 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:29.895610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-15b888d9-5194-4de8-adbe-aace797c6bb8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 08:47:29.895610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:29.895610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:29.895610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:29.895610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:29.895610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:29.895610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:29.895610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:29.895610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:29.895610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:29.895610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:29.895610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:29.895610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:29.895610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:29.895610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:29.895610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:29.896794 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:29.896794 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:29.896794 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:29.896794 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:29.896794 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:29.896794 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:29.896794 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:29.896794 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:29.896794 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:29.896794 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:29.896794 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:29.896794 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:29.896794 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:29.896794 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:29.896794 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:29.896794 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:29.896794 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:29.896794 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:29.897763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:29.897763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:29.897763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:29.897763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:29.897763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:29.897763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:29.897763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:29.897763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:29.897763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 08:47:29.897763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:29.897763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 403/804] 10.222.0.46 () {56 vars in 1117 bytes} [Tue Aug 2 08:47:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:29.898517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b888d9-5194-4de8-adbe-aace797c6bb8 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:29.915059 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:29.927339 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:29.945776 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15d441-1fe4-49c2-a33e-24d6fca20e7e None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:29.980648 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2b4be-9faa-4ff3-a81d-fb76b36b1e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:29.981740 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2b4be-9faa-4ff3-a81d-fb76b36b1e1d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:29.982295 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2b4be-9faa-4ff3-a81d-fb76b36b1e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:29.983092 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2b4be-9faa-4ff3-a81d-fb76b36b1e1d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:29.983796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa2b4be-9faa-4ff3-a81d-fb76b36b1e1d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:29.985250 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa2b4be-9faa-4ff3-a81d-fb76b36b1e1d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:29.994959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-bfa2b4be-9faa-4ff3-a81d-fb76b36b1e1d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 08:47:29.994959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:29.994959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:47:29.994959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:47:29.994959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:47:29.994959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:47:29.994959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:47:29.994959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:29.994959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 08:47:29.994959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:29.994959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:29.994959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:29.994959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:29.994959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:29.994959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:29.994959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:29.996121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:29.996121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:29.996121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:29.996121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:29.996121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:29.996121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:29.996121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:29.996121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:47:29.996121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:47:29.996121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:47:29.996121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:47:29.996121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:29.996121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:29.996121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:29.996121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:29.996121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:29.997190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:29.997190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:29.997190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:29.997190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:29.997190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:29.997190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:29.997190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:29.997190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:29.997190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:29.997190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:29.997190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:29.997190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:47:29.997190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:29.997190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 08:47:29.997190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:29.998291 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 402/805] 10.222.0.46 () {56 vars in 1129 bytes} [Tue Aug 2 08:47:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:29.999076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2b4be-9faa-4ff3-a81d-fb76b36b1e1d None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:30.016622 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b888d9-5194-4de8-adbe-aace797c6bb8 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:30.052947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f02ae-541e-49c2-912c-ecbe58658f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:30.053769 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f02ae-541e-49c2-912c-ecbe58658f88 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:30.054177 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f02ae-541e-49c2-912c-ecbe58658f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:30.054716 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f02ae-541e-49c2-912c-ecbe58658f88 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:30.055476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f02ae-541e-49c2-912c-ecbe58658f88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:30.055791 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f02ae-541e-49c2-912c-ecbe58658f88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:30.057234 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99f02ae-541e-49c2-912c-ecbe58658f88 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:30.075417 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 404/806] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:30.075417 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f02ae-541e-49c2-912c-ecbe58658f88 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:30.116278 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2b4be-9faa-4ff3-a81d-fb76b36b1e1d None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:30.149824 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75f7d9-7b16-4e9c-b2ee-b8b19f2b6dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:30.150491 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75f7d9-7b16-4e9c-b2ee-b8b19f2b6dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:30.150778 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75f7d9-7b16-4e9c-b2ee-b8b19f2b6dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:30.150955 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:30.151043 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75f7d9-7b16-4e9c-b2ee-b8b19f2b6dd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:30.161630 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75f7d9-7b16-4e9c-b2ee-b8b19f2b6dd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:30.163702 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75f7d9-7b16-4e9c-b2ee-b8b19f2b6dd3 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:30.171681 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:30.175742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f02ae-541e-49c2-912c-ecbe58658f88 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:30.178205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-2b75f7d9-7b16-4e9c-b2ee-b8b19f2b6dd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:30.178205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:30.178205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:30.178205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:30.178205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:30.178205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:30.178205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:30.178205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:30.178205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:30.178205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:30.178205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:30.178205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:30.178205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:30.178205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:30.178205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:30.179269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:30.179269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:30.179269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:30.179269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:30.179269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:30.179269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:30.179269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:30.179269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:30.179269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:30.179269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:30.179269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:30.179269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:30.179269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:30.179269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:30.179269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:30.179269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:30.179269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:30.180658 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:30.180658 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:30.180658 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:30.180658 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:30.180658 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 403/807] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:30.180658 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75f7d9-7b16-4e9c-b2ee-b8b19f2b6dd3 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:30.212937 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa79b7-5f94-41a5-b17a-adb63e0df194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:30.213656 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa79b7-5f94-41a5-b17a-adb63e0df194 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:30.214034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa79b7-5f94-41a5-b17a-adb63e0df194 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:30.214336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa79b7-5f94-41a5-b17a-adb63e0df194 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:30.215118 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa79b7-5f94-41a5-b17a-adb63e0df194 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:30.215431 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa79b7-5f94-41a5-b17a-adb63e0df194 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:30.216865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa79b7-5f94-41a5-b17a-adb63e0df194 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:30.234091 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 405/808] 10.222.0.46 () {56 vars in 1149 bytes} [Tue Aug 2 08:47:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:30.234802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa79b7-5f94-41a5-b17a-adb63e0df194 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:30.284364 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75f7d9-7b16-4e9c-b2ee-b8b19f2b6dd3 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:30.317054 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79d965-1475-4713-aab0-68876796f165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:30.317660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79d965-1475-4713-aab0-68876796f165 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:30.318004 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79d965-1475-4713-aab0-68876796f165 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:30.318344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79d965-1475-4713-aab0-68876796f165 None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:30.319020 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79d965-1475-4713-aab0-68876796f165 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:30.319328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79d965-1475-4713-aab0-68876796f165 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:30.320692 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e79d965-1475-4713-aab0-68876796f165 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:30.325591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 404/809] 10.222.0.46 () {56 vars in 1124 bytes} [Tue Aug 2 08:47:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:30.326771 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9a3c0-b633-4dab-a804-060f5cca2883 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:30.327055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9a3c0-b633-4dab-a804-060f5cca2883 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:30.327329 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f9a3c0-b633-4dab-a804-060f5cca2883 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:30.328513 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 405/810] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:30.329195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9a3c0-b633-4dab-a804-060f5cca2883 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:30.337776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa79b7-5f94-41a5-b17a-adb63e0df194 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:30.368533 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:30.373298 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ea936-ed1d-47ac-8bba-a76da131b76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:30.374058 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ea936-ed1d-47ac-8bba-a76da131b76e None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:30.374342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ea936-ed1d-47ac-8bba-a76da131b76e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:30.374619 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93ea936-ed1d-47ac-8bba-a76da131b76e None admin] PATH_INFO: `/v3/projects/4227a5b804c0414c99d407bb1415cf49/users/ff92ef3b9da74ccab3fbc43f852e3ad0/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:30.376612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93ea936-ed1d-47ac-8bba-a76da131b76e None admin] RBAC: Authorizing `identity:create_grant(project_id=4227a5b804c0414c99d407bb1415cf49, user_id=ff92ef3b9da74ccab3fbc43f852e3ad0, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:30.378078 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93ea936-ed1d-47ac-8bba-a76da131b76e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:30.392018 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:30.431375 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f9a3c0-b633-4dab-a804-060f5cca2883 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:30.458595 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 406/811] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:47:30 2022] PUT /identity/v3/projects/4227a5b804c0414c99d407bb1415cf49/users/ff92ef3b9da74ccab3fbc43f852e3ad0/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:47:30.460132 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ea936-ed1d-47ac-8bba-a76da131b76e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:30.464717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b58cd-c6d1-4457-b5a4-3401ecca252f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:30.468219 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b58cd-c6d1-4457-b5a4-3401ecca252f None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:30.468219 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b58cd-c6d1-4457-b5a4-3401ecca252f None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:30.468219 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5b58cd-c6d1-4457-b5a4-3401ecca252f None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:30.468219 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b58cd-c6d1-4457-b5a4-3401ecca252f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:30.468219 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b58cd-c6d1-4457-b5a4-3401ecca252f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:30.469088 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5b58cd-c6d1-4457-b5a4-3401ecca252f None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:30.487140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 406/812] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:30.487140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b58cd-c6d1-4457-b5a4-3401ecca252f None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:30.589335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93ea936-ed1d-47ac-8bba-a76da131b76e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:30.606068 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5b58cd-c6d1-4457-b5a4-3401ecca252f None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:30.633152 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:30.634700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85362284-8508-4af6-88a2-9c75d95b325b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:30.635346 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85362284-8508-4af6-88a2-9c75d95b325b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:30.635630 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85362284-8508-4af6-88a2-9c75d95b325b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:30.635899 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85362284-8508-4af6-88a2-9c75d95b325b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:30.644057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9d82ac-9d0e-4184-a5a8-48a38a42362c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:30.645086 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d82ac-9d0e-4184-a5a8-48a38a42362c None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:30.645385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d82ac-9d0e-4184-a5a8-48a38a42362c None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:30.645665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d82ac-9d0e-4184-a5a8-48a38a42362c None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:30.647168 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d82ac-9d0e-4184-a5a8-48a38a42362c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:30.651190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d82ac-9d0e-4184-a5a8-48a38a42362c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:30.651300 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85362284-8508-4af6-88a2-9c75d95b325b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:30.651393 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9d82ac-9d0e-4184-a5a8-48a38a42362c None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:30.651478 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85362284-8508-4af6-88a2-9c75d95b325b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:30.653830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 407/813] 10.222.0.46 () {56 vars in 1124 bytes} [Tue Aug 2 08:47:30 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:30.655075 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865b53a-f915-47d1-bf25-1aa0525e1e60 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:30.655370 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865b53a-f915-47d1-bf25-1aa0525e1e60 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:30.655653 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865b53a-f915-47d1-bf25-1aa0525e1e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:30.671353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-85362284-8508-4af6-88a2-9c75d95b325b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:30.671353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:30.671353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:30.671353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:30.671353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:30.671353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:30.671353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:30.671353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:30.671353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:30.671353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:30.671353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:30.671353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:30.671353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:30.671353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:30.671353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:30.672637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:30.672637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:30.672637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:30.672637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:30.672637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:30.672637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:30.672637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:30.672637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:30.672637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:30.672637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:30.672637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:30.672637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:30.672637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:30.672637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:30.672637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:30.672637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:30.672637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:30.674023 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:30.674023 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:30.674023 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:30.674023 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:30.674023 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 407/814] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:30.674023 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85362284-8508-4af6-88a2-9c75d95b325b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:30.674466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-3865b53a-f915-47d1-bf25-1aa0525e1e60 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:30.677225 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-3865b53a-f915-47d1-bf25-1aa0525e1e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:30.722044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:30.754116 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 408/815] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:47:30.754975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865b53a-f915-47d1-bf25-1aa0525e1e60 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:30.783921 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85362284-8508-4af6-88a2-9c75d95b325b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:30.821128 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080936f-2f11-40ff-814a-b4bcd31b3fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:30.821781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080936f-2f11-40ff-814a-b4bcd31b3fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:30.822097 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080936f-2f11-40ff-814a-b4bcd31b3fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:30.822381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080936f-2f11-40ff-814a-b4bcd31b3fd5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:30.831014 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3080936f-2f11-40ff-814a-b4bcd31b3fd5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:30.832835 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3080936f-2f11-40ff-814a-b4bcd31b3fd5 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:30.844551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-3080936f-2f11-40ff-814a-b4bcd31b3fd5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:47:30.844551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:30.844551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:30.844551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:30.844551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:30.844551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:30.844551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:30.844551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:30.844551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:30.844551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:30.844551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:30.844551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:30.844551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:30.844551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:30.844551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:30.844551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:30.845567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:30.845567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:30.845567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:30.845567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:30.845567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:30.845567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:30.845567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:30.845567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:30.845567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:30.845567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:30.845567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:30.845567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:30.845567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:30.845567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:30.845567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:30.845567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:30.845567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:30.845567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:30.846598 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:30.846598 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:30.846598 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:30.846598 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:30.846598 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:30.846598 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:30.846598 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:30.846598 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:30.846598 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:47:30.846598 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:30.846598 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 408/816] 10.222.0.46 () {56 vars in 1093 bytes} [Tue Aug 2 08:47:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:30.846598 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080936f-2f11-40ff-814a-b4bcd31b3fd5 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:30.871262 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865b53a-f915-47d1-bf25-1aa0525e1e60 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:30.891406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:30.909698 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a5c61-2f6e-4538-9f29-ac17b488857a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:30.910357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a5c61-2f6e-4538-9f29-ac17b488857a None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:30.910643 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a5c61-2f6e-4538-9f29-ac17b488857a None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:30.910922 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954a5c61-2f6e-4538-9f29-ac17b488857a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:30.911641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a5c61-2f6e-4538-9f29-ac17b488857a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:30.913565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954a5c61-2f6e-4538-9f29-ac17b488857a None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:30.957142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080936f-2f11-40ff-814a-b4bcd31b3fd5 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:30.959598 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:30.989588 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a98e2-929e-457c-90ac-35340457f022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:30.990969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a98e2-929e-457c-90ac-35340457f022 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:30.991322 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a98e2-929e-457c-90ac-35340457f022 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:30.991521 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a98e2-929e-457c-90ac-35340457f022 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:30.992203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a98e2-929e-457c-90ac-35340457f022 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:30.992505 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a98e2-929e-457c-90ac-35340457f022 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:30.993854 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62a98e2-929e-457c-90ac-35340457f022 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:31.010532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 409/817] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 08:47:30 2022] POST /identity/v3/endpoints => generated 346 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:47:31.010532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a5c61-2f6e-4538-9f29-ac17b488857a None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:31.016673 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 409/818] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:31.017573 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a98e2-929e-457c-90ac-35340457f022 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:31.129550 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954a5c61-2f6e-4538-9f29-ac17b488857a None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:31.155814 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a98e2-929e-457c-90ac-35340457f022 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:31.175834 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758ae9a-1c9d-45e6-aba7-165149c1a2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:31.176695 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758ae9a-1c9d-45e6-aba7-165149c1a2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:31.176989 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758ae9a-1c9d-45e6-aba7-165149c1a2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:31.177266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758ae9a-1c9d-45e6-aba7-165149c1a2e1 None admin] PATH_INFO: `/v3/services/network` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:31.179634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3758ae9a-1c9d-45e6-aba7-165149c1a2e1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:31.181506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3758ae9a-1c9d-45e6-aba7-165149c1a2e1 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:31.190994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-3758ae9a-1c9d-45e6-aba7-165149c1a2e1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 08:47:31.190994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:31.190994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:47:31.190994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:47:31.190994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:47:31.190994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:47:31.190994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:47:31.190994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:31.190994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 08:47:31.190994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:31.190994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:31.190994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:31.190994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:31.190994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:31.190994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:31.190994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:31.190994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:31.192041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:31.192041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:31.192041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:31.192041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:31.192041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:31.192041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:31.192041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:47:31.192041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:47:31.192041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:47:31.192041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:47:31.192041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:31.192041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:31.192041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:31.192041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:31.192041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:31.192041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:31.194564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:31.194564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:31.194564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:31.194564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:31.194564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:31.194564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:31.194564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:31.194564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:31.194564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:31.194564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:31.194564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:47:31.194564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:31.194564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 08:47:31.194564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:31.194564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 410/819] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:31 2022] GET /identity/v3/services/network => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:31.196532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758ae9a-1c9d-45e6-aba7-165149c1a2e1 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:31.213837 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85415dca-862f-4be7-8a47-73a83a4617c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:31.214832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85415dca-862f-4be7-8a47-73a83a4617c9 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:31.215124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85415dca-862f-4be7-8a47-73a83a4617c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:31.215405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85415dca-862f-4be7-8a47-73a83a4617c9 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:31.216239 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85415dca-862f-4be7-8a47-73a83a4617c9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:31.216552 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85415dca-862f-4be7-8a47-73a83a4617c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:31.219585 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85415dca-862f-4be7-8a47-73a83a4617c9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:31.233398 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:31.243542 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 410/820] 10.222.0.46 () {56 vars in 1133 bytes} [Tue Aug 2 08:47:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:31.244452 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85415dca-862f-4be7-8a47-73a83a4617c9 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:31.277567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:31.320644 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758ae9a-1c9d-45e6-aba7-165149c1a2e1 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:31.360916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85415dca-862f-4be7-8a47-73a83a4617c9 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:31.368437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed5f2f-b199-483a-bca2-2e74e91cde31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:31.369089 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed5f2f-b199-483a-bca2-2e74e91cde31 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:31.369429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed5f2f-b199-483a-bca2-2e74e91cde31 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:31.369717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ed5f2f-b199-483a-bca2-2e74e91cde31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:31.379361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed5f2f-b199-483a-bca2-2e74e91cde31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:31.381265 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ed5f2f-b199-483a-bca2-2e74e91cde31 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:31.396498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-d3ed5f2f-b199-483a-bca2-2e74e91cde31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:31.396498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:31.396498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:31.396498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:31.396498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:31.396498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:31.396498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:31.396498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:31.396498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:31.396498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:31.396498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:31.396498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:31.396498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:31.396498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:31.396498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:31.397463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:31.397463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:31.397463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:31.397463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:31.397463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:31.397463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:31.397463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:31.397463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:31.397463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:31.397463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:31.397463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:31.397463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:31.397463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:31.397463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:31.397463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:31.397463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:31.397463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:31.398712 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:31.398712 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:31.398712 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:31.398712 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:31.398712 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:31.398712 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:31.398712 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:31.398712 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:31.401929 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 411/821] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:31.401929 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed5f2f-b199-483a-bca2-2e74e91cde31 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:31.410516 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9ee2f-af84-4e47-b1e2-4a9a8aebcc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:31.411225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9ee2f-af84-4e47-b1e2-4a9a8aebcc50 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:31.411574 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9ee2f-af84-4e47-b1e2-4a9a8aebcc50 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:31.412471 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9ee2f-af84-4e47-b1e2-4a9a8aebcc50 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:31.413361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9ee2f-af84-4e47-b1e2-4a9a8aebcc50 None admin] RBAC: Adding query filter params (name=network) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:31.413732 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9ee2f-af84-4e47-b1e2-4a9a8aebcc50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:31.415327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9ee2f-af84-4e47-b1e2-4a9a8aebcc50 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:31.421044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 411/822] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 2 08:47:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:31.421853 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9ee2f-af84-4e47-b1e2-4a9a8aebcc50 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:31.517904 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:31.533523 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9ee2f-af84-4e47-b1e2-4a9a8aebcc50 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:31.533726 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:31.546517 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ed5f2f-b199-483a-bca2-2e74e91cde31 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:31.574541 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d4979-38f9-45d8-b502-4b7d60891c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:31.575341 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d4979-38f9-45d8-b502-4b7d60891c46 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:31.575701 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d4979-38f9-45d8-b502-4b7d60891c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:31.576010 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d4979-38f9-45d8-b502-4b7d60891c46 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:31.577037 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d4979-38f9-45d8-b502-4b7d60891c46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:31.577469 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d4979-38f9-45d8-b502-4b7d60891c46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:31.579031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675d4979-38f9-45d8-b502-4b7d60891c46 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:31.590265 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108f1107-2fcd-4ff1-8fa8-1991d401607c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:31.591177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f1107-2fcd-4ff1-8fa8-1991d401607c None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:31.591641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f1107-2fcd-4ff1-8fa8-1991d401607c None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:31.592081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f1107-2fcd-4ff1-8fa8-1991d401607c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:31.598763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 412/823] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:31.598763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d4979-38f9-45d8-b502-4b7d60891c46 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:31.672915 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108f1107-2fcd-4ff1-8fa8-1991d401607c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:31.675104 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108f1107-2fcd-4ff1-8fa8-1991d401607c None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:31.711189 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d4979-38f9-45d8-b502-4b7d60891c46 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:31.748072 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4207377-8a72-4c29-9e72-ba5a36b255b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:31.748698 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4207377-8a72-4c29-9e72-ba5a36b255b5 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:31.748987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4207377-8a72-4c29-9e72-ba5a36b255b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:31.749269 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4207377-8a72-4c29-9e72-ba5a36b255b5 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:31.749981 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4207377-8a72-4c29-9e72-ba5a36b255b5 None admin] RBAC: Adding query filter params (type=network) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:31.750325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4207377-8a72-4c29-9e72-ba5a36b255b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:31.752012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4207377-8a72-4c29-9e72-ba5a36b255b5 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:31.762374 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 413/824] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 2 08:47:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:31.763256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4207377-8a72-4c29-9e72-ba5a36b255b5 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:31.776878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:31.779703 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:31.790298 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 412/825] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:47:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:47:31.794306 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108f1107-2fcd-4ff1-8fa8-1991d401607c None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:31.872048 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4207377-8a72-4c29-9e72-ba5a36b255b5 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:31.897337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108f1107-2fcd-4ff1-8fa8-1991d401607c None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:31.909587 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf3c35d-5320-474e-8c59-10f73c44e31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:31.910213 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3c35d-5320-474e-8c59-10f73c44e31d None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:31.910620 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3c35d-5320-474e-8c59-10f73c44e31d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:31.910905 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3c35d-5320-474e-8c59-10f73c44e31d None admin] PATH_INFO: `/v3/projects/4227a5b804c0414c99d407bb1415cf49/users/ed8114b7a28741268fdff71c866dbc77/roles/391becda2bf04438b8d1763ab99acf1b` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:31.931277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf3c35d-5320-474e-8c59-10f73c44e31d None admin] RBAC: Authorizing `identity:create_grant(project_id=4227a5b804c0414c99d407bb1415cf49, user_id=ed8114b7a28741268fdff71c866dbc77, role_id=391becda2bf04438b8d1763ab99acf1b)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:31.933161 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf3c35d-5320-474e-8c59-10f73c44e31d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:31.942044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b46f7-125d-475d-a46f-9fe240ec20eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:31.943007 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b46f7-125d-475d-a46f-9fe240ec20eb None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:31.944085 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b46f7-125d-475d-a46f-9fe240ec20eb None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:31.944564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b46f7-125d-475d-a46f-9fe240ec20eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:31.945662 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b46f7-125d-475d-a46f-9fe240ec20eb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:31.947414 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433b46f7-125d-475d-a46f-9fe240ec20eb None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:32.032420 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 414/826] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:47:31 2022] PUT /identity/v3/projects/4227a5b804c0414c99d407bb1415cf49/users/ed8114b7a28741268fdff71c866dbc77/roles/391becda2bf04438b8d1763ab99acf1b => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:47:32.033666 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf3c35d-5320-474e-8c59-10f73c44e31d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:32.043925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 413/827] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 08:47:31 2022] POST /identity/v3/endpoints => generated 337 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:47:32.116974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:32.168406 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf3c35d-5320-474e-8c59-10f73c44e31d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:32.219294 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd79af-5d67-456b-b1f9-42d96aa92f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:32.220184 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd79af-5d67-456b-b1f9-42d96aa92f32 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:32.220569 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd79af-5d67-456b-b1f9-42d96aa92f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:32.220947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd79af-5d67-456b-b1f9-42d96aa92f32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:32.238678 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd79af-5d67-456b-b1f9-42d96aa92f32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:32.241101 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd79af-5d67-456b-b1f9-42d96aa92f32 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:32.263797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-73cd79af-5d67-456b-b1f9-42d96aa92f32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:32.263797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:32.263797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:32.263797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:32.263797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:32.263797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:32.263797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:32.263797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:32.263797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:32.263797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:32.263797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:32.263797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:32.263797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:32.263797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:32.263797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:32.264781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:32.264781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:32.264781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:32.264781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:32.264781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:32.264781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:32.264781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:32.264781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:32.264781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:32.264781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:32.264781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:32.264781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:32.264781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:32.264781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:32.264781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:32.264781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:32.264781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:32.265691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:32.265691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:32.265691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:32.265691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:32.265691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 415/828] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:32.271999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b46f7-125d-475d-a46f-9fe240ec20eb None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:32.333126 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:32.393619 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b46f7-125d-475d-a46f-9fe240ec20eb None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:32.435641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b80e5e-22ab-4658-b0f8-d4f00e2e29bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:32.436642 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b80e5e-22ab-4658-b0f8-d4f00e2e29bf None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:32.437166 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b80e5e-22ab-4658-b0f8-d4f00e2e29bf None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:32.437635 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b80e5e-22ab-4658-b0f8-d4f00e2e29bf None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:32.438541 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b80e5e-22ab-4658-b0f8-d4f00e2e29bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:32.439031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b80e5e-22ab-4658-b0f8-d4f00e2e29bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:32.440582 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b80e5e-22ab-4658-b0f8-d4f00e2e29bf None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:32.459853 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 414/829] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:32.461488 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd79af-5d67-456b-b1f9-42d96aa92f32 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:32.557478 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61e8fa-2be5-41db-995a-0d878c6aacc2 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:32.558154 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61e8fa-2be5-41db-995a-0d878c6aacc2 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:32.558640 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61e8fa-2be5-41db-995a-0d878c6aacc2 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:32.561675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 415/830] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:32.566314 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537b81b-1592-4e5d-a53f-4983eb0dd8cb None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:32.566848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537b81b-1592-4e5d-a53f-4983eb0dd8cb None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:32.567330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537b81b-1592-4e5d-a53f-4983eb0dd8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:32.578991 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd79af-5d67-456b-b1f9-42d96aa92f32 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:32.591406 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:32.591549 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-2537b81b-1592-4e5d-a53f-4983eb0dd8cb None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:32.592546 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-2537b81b-1592-4e5d-a53f-4983eb0dd8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:32.624513 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663f474-96f8-4580-9260-82257d38971f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:32.625930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663f474-96f8-4580-9260-82257d38971f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:32.626664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663f474-96f8-4580-9260-82257d38971f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:32.627171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1663f474-96f8-4580-9260-82257d38971f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:32.638656 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1663f474-96f8-4580-9260-82257d38971f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:32.640970 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1663f474-96f8-4580-9260-82257d38971f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:32.657568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-1663f474-96f8-4580-9260-82257d38971f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:32.657568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:32.657568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:32.657568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:32.657568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:32.657568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:32.657568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:32.657568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:32.657568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:32.657568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:32.657568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:32.657568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:32.657568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:32.657568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:32.657568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:32.658555 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:32.658555 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:32.658555 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:32.658555 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:32.658555 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:32.658555 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:32.658555 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:32.658555 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:32.658555 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:32.658555 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:32.658555 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:32.658555 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:32.658555 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:32.658555 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:32.658555 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:32.658555 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:32.658555 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:32.659437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:32.659437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:32.659437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:32.659437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:32.659437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:32.659437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:32.659437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:32.659437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:32.667417 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 416/831] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:32.669674 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663f474-96f8-4580-9260-82257d38971f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:32.685920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 416/832] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:32 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:47:32.704272 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537b81b-1592-4e5d-a53f-4983eb0dd8cb None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:32.751252 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:32.789740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1663f474-96f8-4580-9260-82257d38971f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:32.815861 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537b81b-1592-4e5d-a53f-4983eb0dd8cb None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:32.827796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45941ba-2c43-4f84-aaaf-42aef9a36701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:32.828659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45941ba-2c43-4f84-aaaf-42aef9a36701 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:32.829208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45941ba-2c43-4f84-aaaf-42aef9a36701 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:32.829729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45941ba-2c43-4f84-aaaf-42aef9a36701 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:32.830716 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45941ba-2c43-4f84-aaaf-42aef9a36701 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:32.831166 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45941ba-2c43-4f84-aaaf-42aef9a36701 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:32.832859 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45941ba-2c43-4f84-aaaf-42aef9a36701 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:32.854437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 417/833] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:32.856398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45941ba-2c43-4f84-aaaf-42aef9a36701 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:32.863148 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989994d-b56b-4af6-8e5d-b29498ef6aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:32.863959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989994d-b56b-4af6-8e5d-b29498ef6aad None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:32.864349 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989994d-b56b-4af6-8e5d-b29498ef6aad None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:32.864729 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989994d-b56b-4af6-8e5d-b29498ef6aad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:32.877035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989994d-b56b-4af6-8e5d-b29498ef6aad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:47:32.883150 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-4989994d-b56b-4af6-8e5d-b29498ef6aad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:47:32.883150 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:32.883150 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:32.883150 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:32.883150 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:32.883150 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:32.883150 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:32.883150 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:32.883150 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:32.883150 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:32.883150 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:32.883150 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:32.883150 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:47:32.883150 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:47:32.883150 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:47:32.883150 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:47:32.884284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:47:32.884284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:47:32.884284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:47:32.884284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:47:32.884284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:32.884284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:32.884284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:32.884284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:32.884284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:32.884284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:32.884284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:32.884284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:32.884284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:32.884284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:32.884284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:32.884284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:32.885299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:32.885299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:32.885299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:32.885299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:32.885299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:47:32.885299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:47:32.885299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:47:32.885299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:47:32.885299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:47:32.885299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:47:32.885299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:47:32.885299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:32.892343 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:32.892670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 417/834] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:47:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:32.894359 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989994d-b56b-4af6-8e5d-b29498ef6aad None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:32.984102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45941ba-2c43-4f84-aaaf-42aef9a36701 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:33.004212 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:33.008759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989994d-b56b-4af6-8e5d-b29498ef6aad None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:33.025587 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffb545-3e51-4415-b2c8-71e2380f866c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:33.026395 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffb545-3e51-4415-b2c8-71e2380f866c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:33.026792 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffb545-3e51-4415-b2c8-71e2380f866c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:33.027332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffb545-3e51-4415-b2c8-71e2380f866c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:33.028651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffb545-3e51-4415-b2c8-71e2380f866c None admin] RBAC: Adding query filter params (role.id=8ce0850e15804f3d811e88cc9841e673, scope.project.id=4227a5b804c0414c99d407bb1415cf49, user.id=3191d804e8dc4b1a85f490319833ec07) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:33.029065 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffb545-3e51-4415-b2c8-71e2380f866c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:33.030914 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffb545-3e51-4415-b2c8-71e2380f866c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:33.042035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 418/835] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:47:32 2022] GET /identity/v3/role_assignments?user.id=3191d804e8dc4b1a85f490319833ec07&scope.project.id=4227a5b804c0414c99d407bb1415cf49&role.id=8ce0850e15804f3d811e88cc9841e673 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:33.058450 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a331f4-b750-49ff-9183-586ee9028a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:33.059429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a331f4-b750-49ff-9183-586ee9028a1c None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:33.059906 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a331f4-b750-49ff-9183-586ee9028a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:33.060327 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a331f4-b750-49ff-9183-586ee9028a1c None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:33.061213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a331f4-b750-49ff-9183-586ee9028a1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:33.061674 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a331f4-b750-49ff-9183-586ee9028a1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:33.063304 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a331f4-b750-49ff-9183-586ee9028a1c None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:33.074663 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 418/836] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:47:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:33.080129 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffb545-3e51-4415-b2c8-71e2380f866c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:33.187840 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:33.197935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffb545-3e51-4415-b2c8-71e2380f866c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:33.234693 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75bf54f-0480-487a-a15d-4ede89d70403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:33.235347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bf54f-0480-487a-a15d-4ede89d70403 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:33.235624 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bf54f-0480-487a-a15d-4ede89d70403 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:33.235906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bf54f-0480-487a-a15d-4ede89d70403 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:33.247370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75bf54f-0480-487a-a15d-4ede89d70403 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:33.248855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75bf54f-0480-487a-a15d-4ede89d70403 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:33.258555 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3037da-7bf5-467a-927e-9514fac8ce74 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:33.258848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3037da-7bf5-467a-927e-9514fac8ce74 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:33.259126 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3037da-7bf5-467a-927e-9514fac8ce74 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:33.260733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 419/837] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:47:33.264699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-e75bf54f-0480-487a-a15d-4ede89d70403 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:33.264699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:33.264699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:33.264699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:33.264699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:33.264699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:33.264699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:33.264699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:33.264699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:33.264699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:33.264699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:33.264699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:33.264699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:33.264699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:33.264699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:33.265601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:33.265601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:33.265601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:33.265601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:33.265601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:33.265601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:33.265601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:33.265601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:33.265601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:33.265601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:33.265601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:33.265601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:33.265601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:33.265601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:33.265601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:33.265601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:33.265601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:33.266619 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:33.266619 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:33.266619 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:33.266619 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:33.266854 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e88d2-9b12-43df-9177-0b36e97cde26 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:33.266854 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e88d2-9b12-43df-9177-0b36e97cde26 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:33.266854 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311e88d2-9b12-43df-9177-0b36e97cde26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:33.267073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 419/838] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:33.273691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75bf54f-0480-487a-a15d-4ede89d70403 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:33.288874 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-311e88d2-9b12-43df-9177-0b36e97cde26 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:33.291302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-311e88d2-9b12-43df-9177-0b36e97cde26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:33.384543 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 420/839] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:47:33.402632 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75bf54f-0480-487a-a15d-4ede89d70403 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:33.406925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e88d2-9b12-43df-9177-0b36e97cde26 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:33.448411 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ce37e-877a-4bb6-be24-53b57e8a42ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:33.448956 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ce37e-877a-4bb6-be24-53b57e8a42ad None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:33.449241 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ce37e-877a-4bb6-be24-53b57e8a42ad None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:33.449519 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ce37e-877a-4bb6-be24-53b57e8a42ad None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:33.450277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141ce37e-877a-4bb6-be24-53b57e8a42ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:33.450822 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141ce37e-877a-4bb6-be24-53b57e8a42ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:33.453232 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141ce37e-877a-4bb6-be24-53b57e8a42ad None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:33.469751 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 420/840] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:33.475495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ce37e-877a-4bb6-be24-53b57e8a42ad None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:33.480946 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:33.515081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:33.525491 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311e88d2-9b12-43df-9177-0b36e97cde26 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:33.565205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b541f1-cb55-45bf-aac1-95f23ffc66f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:33.566277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b541f1-cb55-45bf-aac1-95f23ffc66f0 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:33.566818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b541f1-cb55-45bf-aac1-95f23ffc66f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:33.567438 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b541f1-cb55-45bf-aac1-95f23ffc66f0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:33.568323 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b541f1-cb55-45bf-aac1-95f23ffc66f0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:33.569924 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b541f1-cb55-45bf-aac1-95f23ffc66f0 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:33.580241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-66b541f1-cb55-45bf-aac1-95f23ffc66f0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 08:47:33.580241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:33.580241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:47:33.580241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:47:33.580241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:47:33.580241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:47:33.580241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:47:33.580241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:33.580241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 08:47:33.580241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:33.580241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:33.580241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:33.580241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:33.580241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:33.580241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:33.580241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:33.580241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:33.581280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:33.581280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:33.581280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:33.581280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:33.581280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:33.581280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:33.581280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:47:33.581280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:47:33.581280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:47:33.581280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:47:33.581280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:33.581280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:33.581280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:33.581280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:33.581280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:33.581280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:33.582270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:33.582270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:33.582270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:33.582270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:33.582270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:33.582270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:33.582270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:33.582270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:33.582270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:33.582270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:33.582270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:47:33.582270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:33.582270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 08:47:33.582270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:33.585262 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 421/841] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 2 08:47:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:33.593367 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b541f1-cb55-45bf-aac1-95f23ffc66f0 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:33.607194 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ce37e-877a-4bb6-be24-53b57e8a42ad None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:33.648211 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68c9d5-6a32-4d75-8bcc-e5769d5fbe83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:33.649672 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68c9d5-6a32-4d75-8bcc-e5769d5fbe83 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:33.650202 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68c9d5-6a32-4d75-8bcc-e5769d5fbe83 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:33.651119 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68c9d5-6a32-4d75-8bcc-e5769d5fbe83 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:33.662260 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68c9d5-6a32-4d75-8bcc-e5769d5fbe83 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:33.664278 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68c9d5-6a32-4d75-8bcc-e5769d5fbe83 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:33.682130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-2a68c9d5-6a32-4d75-8bcc-e5769d5fbe83 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 08:47:33.682130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:33.682130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:33.682130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:33.682130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:33.682130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:33.682130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:33.682130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:33.682130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:33.682130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:33.682130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:33.682130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:33.682130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:33.682130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:33.682130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:33.682130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:33.683131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:33.683131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:33.683131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:33.683131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:33.683131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:33.683131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:33.683131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:33.683131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:33.683131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:33.683131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:33.683131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:33.683131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:33.683131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:33.683131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:33.683131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:33.683131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:33.683131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:33.683131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:33.684092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:33.684092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:33.684092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:33.684092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:33.684092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:33.684092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:33.684092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:33.684092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:33.684092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 08:47:33.684092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:33.687256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 421/842] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:47:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:33.692877 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68c9d5-6a32-4d75-8bcc-e5769d5fbe83 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:33.715961 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b541f1-cb55-45bf-aac1-95f23ffc66f0 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:33.754887 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1932c5c-c64d-4818-be33-4de83b081cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:33.756230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1932c5c-c64d-4818-be33-4de83b081cbf None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:33.756447 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1932c5c-c64d-4818-be33-4de83b081cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:33.756728 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1932c5c-c64d-4818-be33-4de83b081cbf None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:33.757482 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1932c5c-c64d-4818-be33-4de83b081cbf None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:33.757809 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1932c5c-c64d-4818-be33-4de83b081cbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:33.759587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1932c5c-c64d-4818-be33-4de83b081cbf None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:33.765126 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 422/843] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 2 08:47:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:33.773927 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1932c5c-c64d-4818-be33-4de83b081cbf None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:33.776256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:33.801132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:33.816224 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68c9d5-6a32-4d75-8bcc-e5769d5fbe83 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:33.855258 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e66f9-3307-4e2f-8e07-39d5ebf7be81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:33.856194 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e66f9-3307-4e2f-8e07-39d5ebf7be81 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:33.856666 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e66f9-3307-4e2f-8e07-39d5ebf7be81 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:33.857087 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e66f9-3307-4e2f-8e07-39d5ebf7be81 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:33.858091 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e66f9-3307-4e2f-8e07-39d5ebf7be81 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:33.858553 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e66f9-3307-4e2f-8e07-39d5ebf7be81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:33.860221 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e66f9-3307-4e2f-8e07-39d5ebf7be81 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:33.880076 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 422/844] 10.222.0.46 () {56 vars in 1135 bytes} [Tue Aug 2 08:47:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:33.886151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e66f9-3307-4e2f-8e07-39d5ebf7be81 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:33.891646 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1932c5c-c64d-4818-be33-4de83b081cbf None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:33.943214 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6de98c-b6fc-44b2-98a2-9460ea7e87cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:33.944441 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6de98c-b6fc-44b2-98a2-9460ea7e87cc None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:33.944825 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6de98c-b6fc-44b2-98a2-9460ea7e87cc None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:33.945213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6de98c-b6fc-44b2-98a2-9460ea7e87cc None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:33.946044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6de98c-b6fc-44b2-98a2-9460ea7e87cc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:33.946449 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6de98c-b6fc-44b2-98a2-9460ea7e87cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:33.947916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6de98c-b6fc-44b2-98a2-9460ea7e87cc None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:33.953922 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 423/845] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 2 08:47:33 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:34.023038 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e66f9-3307-4e2f-8e07-39d5ebf7be81 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:34.048544 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:34.063101 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a1c9f-9ae0-4586-97fd-fd6a2d01382f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:34.063795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a1c9f-9ae0-4586-97fd-fd6a2d01382f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:34.064083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a1c9f-9ae0-4586-97fd-fd6a2d01382f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:34.064359 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a1c9f-9ae0-4586-97fd-fd6a2d01382f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:34.144793 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a1c9f-9ae0-4586-97fd-fd6a2d01382f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:34.147246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a1c9f-9ae0-4586-97fd-fd6a2d01382f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:34.257342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 423/846] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:47:33 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 08:47:34.265931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6c9b8-ae93-4a1d-8d7c-3b94b2b07a77 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:34.265931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6c9b8-ae93-4a1d-8d7c-3b94b2b07a77 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:34.265931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6c9b8-ae93-4a1d-8d7c-3b94b2b07a77 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:34.265931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 424/847] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:34.267229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a1c9f-9ae0-4586-97fd-fd6a2d01382f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:34.268531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf185c8a-2ea2-404a-a62b-99ef0210ea68 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:34.268821 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf185c8a-2ea2-404a-a62b-99ef0210ea68 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:34.269099 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf185c8a-2ea2-404a-a62b-99ef0210ea68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:34.288323 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-bf185c8a-2ea2-404a-a62b-99ef0210ea68 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:34.289445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-bf185c8a-2ea2-404a-a62b-99ef0210ea68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:34.297310 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:34.377778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a1c9f-9ae0-4586-97fd-fd6a2d01382f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:34.390574 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 425/848] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:34 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:47:34.415626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf185c8a-2ea2-404a-a62b-99ef0210ea68 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:34.426802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ac57c-7fb3-4ae7-9638-67fb774bea18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:34.428951 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ac57c-7fb3-4ae7-9638-67fb774bea18 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:34.429723 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ac57c-7fb3-4ae7-9638-67fb774bea18 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:34.430240 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ac57c-7fb3-4ae7-9638-67fb774bea18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:34.432332 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:34.445254 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ac57c-7fb3-4ae7-9638-67fb774bea18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:34.447596 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ac57c-7fb3-4ae7-9638-67fb774bea18 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:34.468462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-006ac57c-7fb3-4ae7-9638-67fb774bea18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:34.468462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:34.468462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:34.468462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:34.468462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:34.468462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:34.468462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:34.468462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:34.468462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:34.468462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:34.468462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:34.468462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:34.468462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:34.468462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:34.468462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:34.469413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:34.469413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:34.469413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:34.469413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:34.469413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:34.469413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:34.469413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:34.469413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:34.469413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:34.469413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:34.469413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:34.469413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:34.469413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:34.469413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:34.469413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:34.469413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:34.469413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:34.470358 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:34.470358 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:34.470358 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:34.470358 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:34.476556 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 424/849] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:34.477437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ac57c-7fb3-4ae7-9638-67fb774bea18 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:34.533406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf185c8a-2ea2-404a-a62b-99ef0210ea68 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:34.564164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:34.570857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb316f-ff2d-4de8-8634-34908b437f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:34.571927 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb316f-ff2d-4de8-8634-34908b437f6d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:34.572423 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb316f-ff2d-4de8-8634-34908b437f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:34.572869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cb316f-ff2d-4de8-8634-34908b437f6d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:34.584755 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cb316f-ff2d-4de8-8634-34908b437f6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:47:34.589700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-54cb316f-ff2d-4de8-8634-34908b437f6d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:34.589700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:34.589700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:34.589700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:34.589700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:34.589700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:34.589700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:34.589700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:34.589700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:34.589700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:34.589700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:34.589700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:34.589700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:47:34.589700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:47:34.589700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:47:34.589700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:47:34.590688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:47:34.590688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:47:34.590688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:47:34.590688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:47:34.590688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:34.590688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:34.590688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:34.590688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:34.590688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:34.590688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:34.590688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:34.590688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:34.590688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:34.590688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:34.590688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:34.590688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:34.591545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:34.591545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:34.591545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:34.591545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:34.591545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:47:34.591545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:47:34.591545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:47:34.591545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:47:34.591545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:47:34.591545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:47:34.591545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:47:34.591545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:34.595862 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 426/850] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 2 08:47:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:34.596656 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ac57c-7fb3-4ae7-9638-67fb774bea18 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:34.597194 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb316f-ff2d-4de8-8634-34908b437f6d None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:34.635604 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72db8d-a7c5-40fa-af60-1654e9b75deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:34.636600 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72db8d-a7c5-40fa-af60-1654e9b75deb None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:34.637210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72db8d-a7c5-40fa-af60-1654e9b75deb None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:34.637696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72db8d-a7c5-40fa-af60-1654e9b75deb None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:34.638626 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72db8d-a7c5-40fa-af60-1654e9b75deb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:34.639145 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72db8d-a7c5-40fa-af60-1654e9b75deb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:34.640728 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72db8d-a7c5-40fa-af60-1654e9b75deb None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:34.658759 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 425/851] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:34.660000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72db8d-a7c5-40fa-af60-1654e9b75deb None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:34.680154 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:34.707098 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cb316f-ff2d-4de8-8634-34908b437f6d None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:34.754937 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7e1f6-bab1-4d8e-a5f5-b76fd8a74407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:34.755634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7e1f6-bab1-4d8e-a5f5-b76fd8a74407 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:34.755912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7e1f6-bab1-4d8e-a5f5-b76fd8a74407 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:34.756184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7e1f6-bab1-4d8e-a5f5-b76fd8a74407 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:34.756917 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7e1f6-bab1-4d8e-a5f5-b76fd8a74407 None admin] RBAC: Adding query filter params (name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:34.757225 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7e1f6-bab1-4d8e-a5f5-b76fd8a74407 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:34.759266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7e1f6-bab1-4d8e-a5f5-b76fd8a74407 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:34.775048 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72db8d-a7c5-40fa-af60-1654e9b75deb None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:34.780908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 427/852] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 2 08:47:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:34.782312 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7e1f6-bab1-4d8e-a5f5-b76fd8a74407 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:34.814358 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0783011-02a8-4fa7-ac74-dabb55cbcca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:34.815650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0783011-02a8-4fa7-ac74-dabb55cbcca9 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:34.816589 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0783011-02a8-4fa7-ac74-dabb55cbcca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:34.817762 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0783011-02a8-4fa7-ac74-dabb55cbcca9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:34.821694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:34.831057 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0783011-02a8-4fa7-ac74-dabb55cbcca9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:34.833398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0783011-02a8-4fa7-ac74-dabb55cbcca9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:34.850046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-a0783011-02a8-4fa7-ac74-dabb55cbcca9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:34.850046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:34.850046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:34.850046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:34.850046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:34.850046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:34.850046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:34.850046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:34.850046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:34.850046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:34.850046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:34.850046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:34.850046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:34.850046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:34.850046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:34.850974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:34.850974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:34.850974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:34.850974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:34.850974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:34.850974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:34.850974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:34.850974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:34.850974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:34.850974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:34.850974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:34.850974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:34.850974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:34.850974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:34.850974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:34.850974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:34.850974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:34.851880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:34.851880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:34.851880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:34.851880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:34.851880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:34.851880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:34.851880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:34.851880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:34.859846 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 426/853] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:34.861106 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0783011-02a8-4fa7-ac74-dabb55cbcca9 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:34.914739 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7e1f6-bab1-4d8e-a5f5-b76fd8a74407 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:34.972129 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3957bd-6e8b-4363-92f6-70790473d4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:34.973775 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3957bd-6e8b-4363-92f6-70790473d4e7 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:34.975018 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3957bd-6e8b-4363-92f6-70790473d4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:34.975533 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3957bd-6e8b-4363-92f6-70790473d4e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:34.979588 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:34.992177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3957bd-6e8b-4363-92f6-70790473d4e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:34.994693 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3957bd-6e8b-4363-92f6-70790473d4e7 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:35.001637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0783011-02a8-4fa7-ac74-dabb55cbcca9 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:35.015208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-ac3957bd-6e8b-4363-92f6-70790473d4e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:35.015208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:35.015208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:35.015208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:35.015208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:35.015208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:35.015208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:35.015208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:35.015208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:35.015208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:35.015208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:35.015208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:35.015208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:35.015208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:35.015208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:35.016170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:35.016170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:35.016170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:35.016170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:35.016170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:35.016170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:35.016170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:35.016170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:35.016170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:35.016170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:35.016170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:35.016170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:35.016170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:35.016170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:35.016170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:35.016170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:35.016170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:35.017075 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:35.017075 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:35.017075 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:35.017075 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:35.027987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 428/854] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:35.029344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3957bd-6e8b-4363-92f6-70790473d4e7 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:35.047528 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e369287-e010-486d-aef0-2f5e463d3190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:35.048839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e369287-e010-486d-aef0-2f5e463d3190 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:35.049724 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e369287-e010-486d-aef0-2f5e463d3190 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:35.049839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e369287-e010-486d-aef0-2f5e463d3190 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:35.051028 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e369287-e010-486d-aef0-2f5e463d3190 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:35.051815 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e369287-e010-486d-aef0-2f5e463d3190 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:35.054124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e369287-e010-486d-aef0-2f5e463d3190 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:35.070048 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 427/855] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:35.075753 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e369287-e010-486d-aef0-2f5e463d3190 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:35.151759 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:35.157486 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3957bd-6e8b-4363-92f6-70790473d4e7 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:35.206023 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd3617-15ba-4122-8dcd-4acea41b6383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:35.206942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd3617-15ba-4122-8dcd-4acea41b6383 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:35.207063 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd3617-15ba-4122-8dcd-4acea41b6383 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:35.207382 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd3617-15ba-4122-8dcd-4acea41b6383 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:35.208124 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd3617-15ba-4122-8dcd-4acea41b6383 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:35.208363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd3617-15ba-4122-8dcd-4acea41b6383 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:35.209761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dd3617-15ba-4122-8dcd-4acea41b6383 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:35.210534 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e369287-e010-486d-aef0-2f5e463d3190 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:35.227095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 429/856] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:35.227095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bc59a-3f9a-4362-b01d-36849c181b77 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:35.227095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bc59a-3f9a-4362-b01d-36849c181b77 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:35.227095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bc59a-3f9a-4362-b01d-36849c181b77 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:35.227095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 430/857] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:35.228751 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bc59a-3f9a-4362-b01d-36849c181b77 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:35.256695 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a153bd5-13b9-4126-8b67-a2ef3ee19f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:35.257314 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a153bd5-13b9-4126-8b67-a2ef3ee19f87 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:35.257592 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a153bd5-13b9-4126-8b67-a2ef3ee19f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:35.257864 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a153bd5-13b9-4126-8b67-a2ef3ee19f87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:35.258686 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a153bd5-13b9-4126-8b67-a2ef3ee19f87 None admin] RBAC: Adding query filter params (role.id=8ce0850e15804f3d811e88cc9841e673, scope.project.id=4227a5b804c0414c99d407bb1415cf49, user.id=ff92ef3b9da74ccab3fbc43f852e3ad0) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:35.259000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a153bd5-13b9-4126-8b67-a2ef3ee19f87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:35.260464 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a153bd5-13b9-4126-8b67-a2ef3ee19f87 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:35.266948 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 428/858] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:47:35 2022] GET /identity/v3/role_assignments?user.id=ff92ef3b9da74ccab3fbc43f852e3ad0&scope.project.id=4227a5b804c0414c99d407bb1415cf49&role.id=8ce0850e15804f3d811e88cc9841e673 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:35.268093 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f93451-4868-4aa3-aad7-1db5cf5b5ee4 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:35.268381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f93451-4868-4aa3-aad7-1db5cf5b5ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:35.268658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f93451-4868-4aa3-aad7-1db5cf5b5ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:35.292302 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-45f93451-4868-4aa3-aad7-1db5cf5b5ee4 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:35.293934 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-45f93451-4868-4aa3-aad7-1db5cf5b5ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:35.308522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:35.371979 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bc59a-3f9a-4362-b01d-36849c181b77 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:35.373189 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 429/859] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:47:35.395922 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f93451-4868-4aa3-aad7-1db5cf5b5ee4 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:35.419093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a58f0-24c9-4294-ae90-0a5507ebffb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:35.420122 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a58f0-24c9-4294-ae90-0a5507ebffb1 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:35.420606 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a58f0-24c9-4294-ae90-0a5507ebffb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:35.421110 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1a58f0-24c9-4294-ae90-0a5507ebffb1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:35.431409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1a58f0-24c9-4294-ae90-0a5507ebffb1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:35.432881 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1a58f0-24c9-4294-ae90-0a5507ebffb1 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:35.446001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-ea1a58f0-24c9-4294-ae90-0a5507ebffb1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 08:47:35.446001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:35.446001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:35.446001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:35.446001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:35.446001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:35.446001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:35.446001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:35.446001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:35.446001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:35.446001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:35.446001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:35.446001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:35.446001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:35.446001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:35.446001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:35.447031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:35.447031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:35.447031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:35.447031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:35.447031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:35.447031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:35.447031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:35.447031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:35.447031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:35.447031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:35.447031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:35.447031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:35.447031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:35.447031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:35.447031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:35.447031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:35.447031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:35.447031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:35.449409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:35.449409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:35.449409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:35.449409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:35.449409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:35.449409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:35.449409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:35.449409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:35.449409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 08:47:35.449409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:35.449409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 431/860] 10.222.0.46 () {56 vars in 1117 bytes} [Tue Aug 2 08:47:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:35.454145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a58f0-24c9-4294-ae90-0a5507ebffb1 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:35.465274 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:35.531820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f93451-4868-4aa3-aad7-1db5cf5b5ee4 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:35.569331 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654dd35f-8ba8-4052-bed7-8eebb3f5ed99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:35.570161 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654dd35f-8ba8-4052-bed7-8eebb3f5ed99 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:35.570506 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654dd35f-8ba8-4052-bed7-8eebb3f5ed99 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:35.570852 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654dd35f-8ba8-4052-bed7-8eebb3f5ed99 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:35.571565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654dd35f-8ba8-4052-bed7-8eebb3f5ed99 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:35.573005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654dd35f-8ba8-4052-bed7-8eebb3f5ed99 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:35.582547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-654dd35f-8ba8-4052-bed7-8eebb3f5ed99 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 08:47:35.582547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:35.582547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:47:35.582547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:47:35.582547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:47:35.582547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:47:35.582547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:47:35.582547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:35.582547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 08:47:35.582547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:35.582547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:35.582547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:35.582547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:35.582547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:35.582547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:35.582547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:35.584318 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:35.584318 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:35.584318 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:35.584318 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:35.584318 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:35.584318 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:35.584318 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:35.584318 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:47:35.584318 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:47:35.584318 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:47:35.584318 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:47:35.584318 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:35.584318 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:35.584318 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:35.584318 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:35.584318 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:35.585209 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:35.585209 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:35.585209 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:35.585209 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:35.585209 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:35.585209 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:35.585209 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:35.585209 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:35.585209 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:35.585209 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:35.585209 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:35.585209 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:47:35.585209 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:35.585209 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 08:47:35.585209 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:35.589142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 430/861] 10.222.0.46 () {56 vars in 1132 bytes} [Tue Aug 2 08:47:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:35.590383 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654dd35f-8ba8-4052-bed7-8eebb3f5ed99 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:35.603221 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1a58f0-24c9-4294-ae90-0a5507ebffb1 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:35.618472 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:35.648590 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb35db6-8e47-46b7-a107-60b39e4aa449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:35.650109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb35db6-8e47-46b7-a107-60b39e4aa449 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:35.650409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb35db6-8e47-46b7-a107-60b39e4aa449 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:35.650681 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb35db6-8e47-46b7-a107-60b39e4aa449 None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:35.651485 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb35db6-8e47-46b7-a107-60b39e4aa449 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:35.651882 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb35db6-8e47-46b7-a107-60b39e4aa449 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:35.653394 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb35db6-8e47-46b7-a107-60b39e4aa449 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:35.677389 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 432/862] 10.222.0.46 () {56 vars in 1149 bytes} [Tue Aug 2 08:47:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:35.682965 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb35db6-8e47-46b7-a107-60b39e4aa449 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:35.713997 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654dd35f-8ba8-4052-bed7-8eebb3f5ed99 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:35.729523 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:35.753940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3469f4c-96ad-43fe-8bd7-e04a4fde5e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:35.755044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3469f4c-96ad-43fe-8bd7-e04a4fde5e0e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:35.765176 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3469f4c-96ad-43fe-8bd7-e04a4fde5e0e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:35.765176 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3469f4c-96ad-43fe-8bd7-e04a4fde5e0e None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:35.765176 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3469f4c-96ad-43fe-8bd7-e04a4fde5e0e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:35.765176 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3469f4c-96ad-43fe-8bd7-e04a4fde5e0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:35.765176 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3469f4c-96ad-43fe-8bd7-e04a4fde5e0e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:35.765176 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 431/863] 10.222.0.46 () {56 vars in 1126 bytes} [Tue Aug 2 08:47:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:35.769540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3469f4c-96ad-43fe-8bd7-e04a4fde5e0e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:35.788270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb35db6-8e47-46b7-a107-60b39e4aa449 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:35.825738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abc693c-d6a9-4046-922d-c9e927c482c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:35.826395 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc693c-d6a9-4046-922d-c9e927c482c4 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:35.826680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc693c-d6a9-4046-922d-c9e927c482c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:35.826960 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abc693c-d6a9-4046-922d-c9e927c482c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:35.878304 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:35.889229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3469f4c-96ad-43fe-8bd7-e04a4fde5e0e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:35.898990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc693c-d6a9-4046-922d-c9e927c482c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:35.900516 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abc693c-d6a9-4046-922d-c9e927c482c4 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:35.929203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed693d-1f6b-4844-9d67-bc338550c3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:35.930065 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed693d-1f6b-4844-9d67-bc338550c3e1 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:35.930454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed693d-1f6b-4844-9d67-bc338550c3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:35.930867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed693d-1f6b-4844-9d67-bc338550c3e1 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:35.931655 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed693d-1f6b-4844-9d67-bc338550c3e1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:35.932103 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed693d-1f6b-4844-9d67-bc338550c3e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:35.938869 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed693d-1f6b-4844-9d67-bc338550c3e1 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:35.945205 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 432/864] 10.222.0.46 () {56 vars in 1126 bytes} [Tue Aug 2 08:47:35 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:36.011658 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 433/865] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:47:35 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 08:47:36.019056 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed693d-1f6b-4844-9d67-bc338550c3e1 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:36.071439 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:36.128360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c686e5e-7bd2-4bef-bf81-0559ef75f333 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:36.128645 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c686e5e-7bd2-4bef-bf81-0559ef75f333 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:36.128921 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c686e5e-7bd2-4bef-bf81-0559ef75f333 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:36.130501 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 434/866] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:36.136514 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed693d-1f6b-4844-9d67-bc338550c3e1 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:36.136901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246f77b-566b-4d5c-9581-406baa735948 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:36.137185 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246f77b-566b-4d5c-9581-406baa735948 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:36.137457 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246f77b-566b-4d5c-9581-406baa735948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:36.171450 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-3246f77b-566b-4d5c-9581-406baa735948 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:36.172633 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-3246f77b-566b-4d5c-9581-406baa735948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:36.177588 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71ef9a-dc6f-4385-8e67-12c4efe9d6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:36.178346 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71ef9a-dc6f-4385-8e67-12c4efe9d6fc None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:36.178734 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71ef9a-dc6f-4385-8e67-12c4efe9d6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:36.179119 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71ef9a-dc6f-4385-8e67-12c4efe9d6fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:36.193998 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b71ef9a-dc6f-4385-8e67-12c4efe9d6fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:36.196394 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b71ef9a-dc6f-4385-8e67-12c4efe9d6fc None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:36.216327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-9b71ef9a-dc6f-4385-8e67-12c4efe9d6fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:36.216327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:36.216327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:36.216327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:36.216327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:36.216327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:36.216327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:36.216327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:36.216327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:36.216327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:36.216327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:36.216327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:36.216327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:36.216327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:36.216327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:36.217476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:36.217476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:36.217476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:36.217476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:36.217476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:36.217476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:36.217476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:36.217476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:36.217476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:36.217476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:36.217476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:36.217476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:36.217476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:36.217476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:36.217476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:36.217476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:36.217476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:36.218630 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:36.218630 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:36.218630 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:36.218630 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:36.221907 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 433/867] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:36.225193 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71ef9a-dc6f-4385-8e67-12c4efe9d6fc None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:36.242092 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:36.250777 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 435/868] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:47:36.273374 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246f77b-566b-4d5c-9581-406baa735948 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:36.346484 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:36.359005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71ef9a-dc6f-4385-8e67-12c4efe9d6fc None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:36.382596 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246f77b-566b-4d5c-9581-406baa735948 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:36.403830 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95b082-85d1-4bfb-a7a0-c163a37f4d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:36.404692 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95b082-85d1-4bfb-a7a0-c163a37f4d8b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:36.405676 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95b082-85d1-4bfb-a7a0-c163a37f4d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:36.406140 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb95b082-85d1-4bfb-a7a0-c163a37f4d8b None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:36.407265 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95b082-85d1-4bfb-a7a0-c163a37f4d8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:36.407723 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95b082-85d1-4bfb-a7a0-c163a37f4d8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:36.410211 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb95b082-85d1-4bfb-a7a0-c163a37f4d8b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:36.421411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb08a4-9f4d-45f4-aba0-b1889dc16b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:36.422246 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb08a4-9f4d-45f4-aba0-b1889dc16b85 None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:36.422912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb08a4-9f4d-45f4-aba0-b1889dc16b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:36.422912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb08a4-9f4d-45f4-aba0-b1889dc16b85 None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:36.423775 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb08a4-9f4d-45f4-aba0-b1889dc16b85 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:36.425327 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb08a4-9f4d-45f4-aba0-b1889dc16b85 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:36.431328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 434/869] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:36.431932 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95b082-85d1-4bfb-a7a0-c163a37f4d8b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:36.504078 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 436/870] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 2 08:47:36 2022] POST /identity/v3/services => generated 251 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 08:47:36.550043 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb95b082-85d1-4bfb-a7a0-c163a37f4d8b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:36.600001 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05da17e-ffb2-4fb5-93e7-3a35018109aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:36.600730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05da17e-ffb2-4fb5-93e7-3a35018109aa None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:36.601021 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05da17e-ffb2-4fb5-93e7-3a35018109aa None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:36.601302 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05da17e-ffb2-4fb5-93e7-3a35018109aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:36.609251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05da17e-ffb2-4fb5-93e7-3a35018109aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:36.610754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05da17e-ffb2-4fb5-93e7-3a35018109aa None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:36.623664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-d05da17e-ffb2-4fb5-93e7-3a35018109aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:36.623664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:36.623664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:36.623664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:36.623664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:36.623664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:36.623664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:36.623664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:36.623664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:36.623664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:36.623664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:36.623664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:36.623664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:36.623664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:36.623664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:36.624641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:36.624641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:36.624641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:36.624641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:36.624641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:36.624641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:36.624641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:36.624641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:36.624641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:36.624641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:36.624641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:36.624641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:36.624641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:36.624641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:36.624641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:36.624641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:36.624641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:36.625548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:36.625548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:36.625548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:36.625548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:36.625548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:36.625548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:36.625548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:36.625548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:36.625548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 435/871] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:36.633306 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb08a4-9f4d-45f4-aba0-b1889dc16b85 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:36.697412 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:36.744508 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb08a4-9f4d-45f4-aba0-b1889dc16b85 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:36.779914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872f222a-0a92-4c46-8992-4a795446b3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:36.780925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f222a-0a92-4c46-8992-4a795446b3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:36.781410 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f222a-0a92-4c46-8992-4a795446b3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:36.781913 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872f222a-0a92-4c46-8992-4a795446b3e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:36.782868 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f222a-0a92-4c46-8992-4a795446b3e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:36.783380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f222a-0a92-4c46-8992-4a795446b3e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:36.785065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872f222a-0a92-4c46-8992-4a795446b3e6 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:36.801527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 437/872] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:36.808924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05da17e-ffb2-4fb5-93e7-3a35018109aa None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:36.823862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:36.919428 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05da17e-ffb2-4fb5-93e7-3a35018109aa None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:36.964287 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe78c16-58d2-4774-b00e-773d977a4d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:36.965202 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe78c16-58d2-4774-b00e-773d977a4d1a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:36.965692 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe78c16-58d2-4774-b00e-773d977a4d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:36.966224 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe78c16-58d2-4774-b00e-773d977a4d1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:36.967241 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe78c16-58d2-4774-b00e-773d977a4d1a None admin] RBAC: Adding query filter params (role.id=391becda2bf04438b8d1763ab99acf1b, scope.project.id=4227a5b804c0414c99d407bb1415cf49, user.id=ed8114b7a28741268fdff71c866dbc77) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:36.967757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe78c16-58d2-4774-b00e-773d977a4d1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:36.969385 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe78c16-58d2-4774-b00e-773d977a4d1a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:36.976196 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 436/873] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:47:36 2022] GET /identity/v3/role_assignments?user.id=ed8114b7a28741268fdff71c866dbc77&scope.project.id=4227a5b804c0414c99d407bb1415cf49&role.id=391becda2bf04438b8d1763ab99acf1b => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:37.539554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845af6b-3456-4c02-926b-d2b9dd5b0f5a None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:37.540274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845af6b-3456-4c02-926b-d2b9dd5b0f5a None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:37.540766 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3845af6b-3456-4c02-926b-d2b9dd5b0f5a None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:37.542656 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 438/874] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:37.548273 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cc8d3-b471-49c9-ab34-347ef2e0bb1b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:37.548812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cc8d3-b471-49c9-ab34-347ef2e0bb1b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:37.549292 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cc8d3-b471-49c9-ab34-347ef2e0bb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:37.571923 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-6e1cc8d3-b471-49c9-ab34-347ef2e0bb1b None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:37.573715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6e1cc8d3-b471-49c9-ab34-347ef2e0bb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:37.645739 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 437/875] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:37 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:47:37.668525 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3845af6b-3456-4c02-926b-d2b9dd5b0f5a None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:37.779082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3845af6b-3456-4c02-926b-d2b9dd5b0f5a None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:37.802878 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:37.815615 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89786419-c333-40c8-b9d7-4f0e9edc29b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:37.816262 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89786419-c333-40c8-b9d7-4f0e9edc29b6 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:37.830200 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89786419-c333-40c8-b9d7-4f0e9edc29b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:37.830200 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89786419-c333-40c8-b9d7-4f0e9edc29b6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:37.830200 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89786419-c333-40c8-b9d7-4f0e9edc29b6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:37.830200 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89786419-c333-40c8-b9d7-4f0e9edc29b6 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:37.830200 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-89786419-c333-40c8-b9d7-4f0e9edc29b6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:47:37.830200 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:37.830200 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:47:37.830200 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:47:37.830200 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:47:37.830791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:47:37.830791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:47:37.830791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:37.830791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:47:37.830791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:37.830791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:37.830791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:37.830791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:37.830791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:37.830791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:37.830791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:37.830791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:37.830791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:37.830791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:37.830791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:37.830791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:37.830791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:37.831759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:37.831759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:47:37.831759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:47:37.831759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:47:37.831759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:47:37.831759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:37.831759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:37.831759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:37.831759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:37.831759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:37.831759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:37.831759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:37.831759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:37.831759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:37.831759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:37.831759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:37.831759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:37.832661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:37.832661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:37.832661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:37.832661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:37.832661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:47:37.832661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:37.832661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:47:37.832661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:37.832661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 439/876] 10.222.0.46 () {56 vars in 1126 bytes} [Tue Aug 2 08:47:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:37.837923 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1cc8d3-b471-49c9-ab34-347ef2e0bb1b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:37.840246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:37.954707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1cc8d3-b471-49c9-ab34-347ef2e0bb1b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:37.991353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d5b86-e3cc-4b40-adf1-66eda99e498d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:37.992339 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d5b86-e3cc-4b40-adf1-66eda99e498d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:37.992802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d5b86-e3cc-4b40-adf1-66eda99e498d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:37.993257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d5b86-e3cc-4b40-adf1-66eda99e498d None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:37.994206 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d5b86-e3cc-4b40-adf1-66eda99e498d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:37.994723 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d5b86-e3cc-4b40-adf1-66eda99e498d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:37.996256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d5b86-e3cc-4b40-adf1-66eda99e498d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:38.001587 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 438/877] 10.222.0.46 () {56 vars in 1122 bytes} [Tue Aug 2 08:47:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:38.008000 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89786419-c333-40c8-b9d7-4f0e9edc29b6 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:38.089059 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae2eab-fcb2-43d6-80df-0c213aac2e3e None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:38.089257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae2eab-fcb2-43d6-80df-0c213aac2e3e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:38.089730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae2eab-fcb2-43d6-80df-0c213aac2e3e None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:38.091344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 439/878] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:38.096145 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb42f4a-9c79-4ad3-9a12-61ce118aac7a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:38.096619 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb42f4a-9c79-4ad3-9a12-61ce118aac7a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:38.096986 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb42f4a-9c79-4ad3-9a12-61ce118aac7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:38.116187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-0fb42f4a-9c79-4ad3-9a12-61ce118aac7a None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:38.117310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89786419-c333-40c8-b9d7-4f0e9edc29b6 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:38.117478 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0fb42f4a-9c79-4ad3-9a12-61ce118aac7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:38.158122 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed912a89-a389-4883-b6b7-3ae1a5b14939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:38.159413 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed912a89-a389-4883-b6b7-3ae1a5b14939 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:38.159413 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed912a89-a389-4883-b6b7-3ae1a5b14939 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:38.159413 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed912a89-a389-4883-b6b7-3ae1a5b14939 None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:38.160053 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed912a89-a389-4883-b6b7-3ae1a5b14939 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:38.160368 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed912a89-a389-4883-b6b7-3ae1a5b14939 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:38.163331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed912a89-a389-4883-b6b7-3ae1a5b14939 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:38.168761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 440/879] 10.222.0.46 () {56 vars in 1122 bytes} [Tue Aug 2 08:47:38 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:38.193467 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:38.202600 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 440/880] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:38 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:47:38.228660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed912a89-a389-4883-b6b7-3ae1a5b14939 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:38.279994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:38.342896 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed912a89-a389-4883-b6b7-3ae1a5b14939 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:38.380242 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe80cc8-dd93-45a3-a751-0fad31d32a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:38.381418 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe80cc8-dd93-45a3-a751-0fad31d32a79 None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:38.381699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe80cc8-dd93-45a3-a751-0fad31d32a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:38.382000 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe80cc8-dd93-45a3-a751-0fad31d32a79 None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:38.382720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe80cc8-dd93-45a3-a751-0fad31d32a79 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:38.384078 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe80cc8-dd93-45a3-a751-0fad31d32a79 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:38.512658 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 441/881] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 2 08:47:38 2022] POST /identity/v3/services => generated 269 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:47:38.645586 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0e270-bf55-43a0-b99f-8332494c12f0 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:38.645768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0e270-bf55-43a0-b99f-8332494c12f0 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:38.646207 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0e270-bf55-43a0-b99f-8332494c12f0 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:38.647841 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 441/882] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:38.653438 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98186e-0733-4d9d-befa-16d9559f4f22 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:38.653740 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98186e-0733-4d9d-befa-16d9559f4f22 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:38.654048 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc98186e-0733-4d9d-befa-16d9559f4f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:38.675194 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-fc98186e-0733-4d9d-befa-16d9559f4f22 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:38.676397 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-fc98186e-0733-4d9d-befa-16d9559f4f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:38.760350 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 442/883] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:38 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:47:38.778791 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0e270-bf55-43a0-b99f-8332494c12f0 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:38.885704 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0e270-bf55-43a0-b99f-8332494c12f0 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:38.921865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2898c7d2-cca8-4157-a1ef-cc8f4e2e5e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:38.922544 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898c7d2-cca8-4157-a1ef-cc8f4e2e5e51 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:38.922826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898c7d2-cca8-4157-a1ef-cc8f4e2e5e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:38.923099 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898c7d2-cca8-4157-a1ef-cc8f4e2e5e51 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:38.923735 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2898c7d2-cca8-4157-a1ef-cc8f4e2e5e51 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:38.925099 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2898c7d2-cca8-4157-a1ef-cc8f4e2e5e51 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:38.935440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-2898c7d2-cca8-4157-a1ef-cc8f4e2e5e51 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 08:47:38.935440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:38.935440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:47:38.935440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:47:38.935440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:47:38.935440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:47:38.935440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:47:38.935440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:38.935440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 08:47:38.935440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:38.935440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:38.935440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:38.935440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:38.935440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:38.935440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:38.935440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:38.935440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:38.936454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:38.936454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:38.936454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:38.936454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:38.936454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:38.936454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:38.936454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:47:38.936454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:47:38.936454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:47:38.936454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:47:38.936454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:38.936454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:38.936454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:38.936454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:38.936454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:38.936454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:38.937325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:38.937325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:38.937325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:38.937325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:38.937325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:38.937325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:38.937325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:38.937325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:38.937325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:38.937325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:38.937325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:47:38.937325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:38.937325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 08:47:38.937325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:38.937325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 442/884] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 2 08:47:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:38.945473 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98186e-0733-4d9d-befa-16d9559f4f22 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:38.977535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:39.053556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc98186e-0733-4d9d-befa-16d9559f4f22 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:39.090848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f42ac-fcea-4423-a5d4-714b45af4715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:39.091754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f42ac-fcea-4423-a5d4-714b45af4715 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:39.092189 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f42ac-fcea-4423-a5d4-714b45af4715 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:39.092616 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f42ac-fcea-4423-a5d4-714b45af4715 None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:39.093442 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f42ac-fcea-4423-a5d4-714b45af4715 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:39.093918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f42ac-fcea-4423-a5d4-714b45af4715 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:39.095434 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f42ac-fcea-4423-a5d4-714b45af4715 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:39.100712 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 443/885] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 2 08:47:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:39.106343 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2898c7d2-cca8-4157-a1ef-cc8f4e2e5e51 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:39.151394 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b26db-ae02-42b9-939f-8bb9f752126f None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:39.151722 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b26db-ae02-42b9-939f-8bb9f752126f None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:39.151959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b26db-ae02-42b9-939f-8bb9f752126f None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:39.153565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 444/886] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:39.158340 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459e309-0d95-416f-99e1-1d60346f000d None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:39.158629 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459e309-0d95-416f-99e1-1d60346f000d None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:39.158916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459e309-0d95-416f-99e1-1d60346f000d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:39.174322 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:39.177951 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-5459e309-0d95-416f-99e1-1d60346f000d None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:39.179158 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-5459e309-0d95-416f-99e1-1d60346f000d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:39.229986 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2898c7d2-cca8-4157-a1ef-cc8f4e2e5e51 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:39.274737 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 445/887] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:39 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:47:39.282807 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36df6e3-836c-4613-874c-bc6478d58e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:39.283429 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36df6e3-836c-4613-874c-bc6478d58e1c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:39.283709 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36df6e3-836c-4613-874c-bc6478d58e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:39.283978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36df6e3-836c-4613-874c-bc6478d58e1c None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:39.284652 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36df6e3-836c-4613-874c-bc6478d58e1c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:39.284953 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36df6e3-836c-4613-874c-bc6478d58e1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:39.286342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36df6e3-836c-4613-874c-bc6478d58e1c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:39.292376 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 443/888] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 2 08:47:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:39.296995 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459e309-0d95-416f-99e1-1d60346f000d None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:39.298893 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36df6e3-836c-4613-874c-bc6478d58e1c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:39.333081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:39.417776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36df6e3-836c-4613-874c-bc6478d58e1c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:39.436508 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459e309-0d95-416f-99e1-1d60346f000d None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:39.444351 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:39.457272 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffea4e2-8b70-4946-9a06-00af7c727934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:39.457912 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffea4e2-8b70-4946-9a06-00af7c727934 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:39.458197 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffea4e2-8b70-4946-9a06-00af7c727934 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:39.458547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffea4e2-8b70-4946-9a06-00af7c727934 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:39.459305 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffea4e2-8b70-4946-9a06-00af7c727934 None admin] RBAC: Adding query filter params (interface=public, service_id=4f54b98ceee742ef8140017f70f04150, region_id=RegionOne) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:39.459721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffea4e2-8b70-4946-9a06-00af7c727934 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:39.461238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffea4e2-8b70-4946-9a06-00af7c727934 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:39.472284 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 444/889] 10.222.0.46 () {56 vars in 1251 bytes} [Tue Aug 2 08:47:39 2022] GET /identity/v3/endpoints?service_id=4f54b98ceee742ef8140017f70f04150&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:39.473828 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffea4e2-8b70-4946-9a06-00af7c727934 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:39.480651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c9949-8452-4138-b745-cfaea1acd324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:39.481490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c9949-8452-4138-b745-cfaea1acd324 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:39.481951 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c9949-8452-4138-b745-cfaea1acd324 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:39.482378 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c9949-8452-4138-b745-cfaea1acd324 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:39.497857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c9949-8452-4138-b745-cfaea1acd324 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:47:39.504575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-289c9949-8452-4138-b745-cfaea1acd324 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 08:47:39.504575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:39.504575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:39.504575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:39.504575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:39.504575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:39.504575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:39.504575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:39.504575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:39.504575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:39.504575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:39.504575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:39.504575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:47:39.504575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:47:39.504575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:47:39.504575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:47:39.505782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:47:39.505782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:47:39.505782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:47:39.505782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:47:39.505782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:39.505782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:39.505782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:39.505782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:39.505782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:39.505782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:39.505782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:39.505782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:39.505782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:39.505782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:39.505782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:39.505782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:39.506877 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:39.506877 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:39.506877 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:39.506877 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:39.506877 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:47:39.506877 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:47:39.506877 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:47:39.506877 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:47:39.506877 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:47:39.506877 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:47:39.506877 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 08:47:39.506877 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:39.514224 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 446/890] 10.222.0.46 () {56 vars in 1120 bytes} [Tue Aug 2 08:47:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:39.515913 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c9949-8452-4138-b745-cfaea1acd324 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:39.599459 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffea4e2-8b70-4946-9a06-00af7c727934 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:39.621767 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:39.642167 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c9949-8452-4138-b745-cfaea1acd324 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:39.645850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f45c4-1d6c-4cab-a4d4-38e885f54c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:39.646733 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f45c4-1d6c-4cab-a4d4-38e885f54c99 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:39.647189 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f45c4-1d6c-4cab-a4d4-38e885f54c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:39.647617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f45c4-1d6c-4cab-a4d4-38e885f54c99 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:39.648422 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084f45c4-1d6c-4cab-a4d4-38e885f54c99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:39.650093 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084f45c4-1d6c-4cab-a4d4-38e885f54c99 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:39.658722 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 445/891] 10.222.0.46 () {56 vars in 1087 bytes} [Tue Aug 2 08:47:39 2022] GET /identity/v3/services => generated 1268 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 08:47:39.691669 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f53336-e621-452b-9f97-63629213d90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:39.692221 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f53336-e621-452b-9f97-63629213d90d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:39.692505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f53336-e621-452b-9f97-63629213d90d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:39.692781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f53336-e621-452b-9f97-63629213d90d None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:39.693540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f53336-e621-452b-9f97-63629213d90d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:39.693847 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f53336-e621-452b-9f97-63629213d90d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:39.695359 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f53336-e621-452b-9f97-63629213d90d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:39.705568 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 447/892] 10.222.0.46 () {56 vars in 1115 bytes} [Tue Aug 2 08:47:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:39.711218 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f45c4-1d6c-4cab-a4d4-38e885f54c99 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:39.775711 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:39.830745 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f45c4-1d6c-4cab-a4d4-38e885f54c99 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:39.867941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ca5a7-77b0-4587-974a-2e43f0ad3e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:39.868591 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ca5a7-77b0-4587-974a-2e43f0ad3e6a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:39.868876 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ca5a7-77b0-4587-974a-2e43f0ad3e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:39.869166 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ca5a7-77b0-4587-974a-2e43f0ad3e6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:39.882065 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ca5a7-77b0-4587-974a-2e43f0ad3e6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:39.883516 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ca5a7-77b0-4587-974a-2e43f0ad3e6a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:39.898405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-d88ca5a7-77b0-4587-974a-2e43f0ad3e6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:39.898405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:39.898405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:39.898405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:39.898405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:39.898405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:39.898405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:39.898405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:39.898405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:39.898405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:39.898405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:39.898405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:39.898405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:39.898405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:39.898405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:39.899361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:39.899361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:39.899361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:39.899361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:39.899361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:39.899361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:39.899361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:39.899361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:39.899361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:39.899361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:39.899361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:39.899361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:39.899361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:39.899361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:39.899361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:39.899361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:39.899361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:39.900660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:39.900660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:39.900660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:39.900660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:39.900660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 446/893] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:39.905452 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f53336-e621-452b-9f97-63629213d90d None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:40.023593 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f53336-e621-452b-9f97-63629213d90d None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:40.059706 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b581a22-a23e-403b-8a19-59e2800ab370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:40.060333 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b581a22-a23e-403b-8a19-59e2800ab370 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:40.060617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b581a22-a23e-403b-8a19-59e2800ab370 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:40.060898 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b581a22-a23e-403b-8a19-59e2800ab370 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:40.061592 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b581a22-a23e-403b-8a19-59e2800ab370 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:40.061913 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b581a22-a23e-403b-8a19-59e2800ab370 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:40.063299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b581a22-a23e-403b-8a19-59e2800ab370 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:40.076951 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 448/894] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:40.082377 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ca5a7-77b0-4587-974a-2e43f0ad3e6a None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:40.189457 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ca5a7-77b0-4587-974a-2e43f0ad3e6a None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:40.191983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:40.228351 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894b85b-8753-44a6-ac56-c5d06ad7068a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:40.229008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894b85b-8753-44a6-ac56-c5d06ad7068a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:40.229291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894b85b-8753-44a6-ac56-c5d06ad7068a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:40.229568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894b85b-8753-44a6-ac56-c5d06ad7068a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:40.238380 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894b85b-8753-44a6-ac56-c5d06ad7068a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:40.239871 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894b85b-8753-44a6-ac56-c5d06ad7068a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:40.251758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-3894b85b-8753-44a6-ac56-c5d06ad7068a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 08:47:40.251758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:40.251758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:40.251758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:40.251758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:40.251758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:40.251758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:40.251758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:40.251758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:40.251758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:40.251758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:40.251758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:40.251758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:40.251758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:40.251758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:40.251758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:40.252714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:40.252714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:40.252714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:40.252714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:40.252714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:40.252714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:40.252714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:40.252714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:40.252714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:40.252714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:40.252714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:40.252714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:40.252714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:40.252714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:40.252714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:40.252714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:40.252714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:40.252714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:40.254296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:40.254296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:40.254296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:40.254296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:40.254296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:40.254296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:40.254296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:40.254296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:40.254296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 08:47:40.254296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:40.254296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 447/895] 10.222.0.46 () {56 vars in 1117 bytes} [Tue Aug 2 08:47:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:40.259569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b581a22-a23e-403b-8a19-59e2800ab370 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:40.269246 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:40.326764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4351035-db01-4ea7-8c2e-fb9210bad1b8 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:40.326956 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4351035-db01-4ea7-8c2e-fb9210bad1b8 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:40.327240 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4351035-db01-4ea7-8c2e-fb9210bad1b8 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:40.328849 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 448/896] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:40.335655 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21d40e-ecc6-480a-8655-d9dfb07c88ac None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:40.335948 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21d40e-ecc6-480a-8655-d9dfb07c88ac None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:40.336225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d21d40e-ecc6-480a-8655-d9dfb07c88ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:40.354669 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-2d21d40e-ecc6-480a-8655-d9dfb07c88ac None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:40.355851 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2d21d40e-ecc6-480a-8655-d9dfb07c88ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:40.369812 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b581a22-a23e-403b-8a19-59e2800ab370 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:40.416115 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cafda8-8cea-46d6-9a25-868531c0ce37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:40.416738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cafda8-8cea-46d6-9a25-868531c0ce37 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:40.417026 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cafda8-8cea-46d6-9a25-868531c0ce37 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:40.417307 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cafda8-8cea-46d6-9a25-868531c0ce37 None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:40.418131 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cafda8-8cea-46d6-9a25-868531c0ce37 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:40.418444 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cafda8-8cea-46d6-9a25-868531c0ce37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:40.419992 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cafda8-8cea-46d6-9a25-868531c0ce37 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:40.439192 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 449/897] 10.222.0.46 () {56 vars in 1149 bytes} [Tue Aug 2 08:47:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:40.440347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 449/898] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:40 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:47:40.446944 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cafda8-8cea-46d6-9a25-868531c0ce37 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:40.465263 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21d40e-ecc6-480a-8655-d9dfb07c88ac None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:40.525251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:40.567398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:40.575389 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d21d40e-ecc6-480a-8655-d9dfb07c88ac None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:40.577436 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cafda8-8cea-46d6-9a25-868531c0ce37 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:40.611058 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2dee60-4732-46d4-9143-5d136ac79e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:40.611679 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2dee60-4732-46d4-9143-5d136ac79e69 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:40.611964 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2dee60-4732-46d4-9143-5d136ac79e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:40.612242 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2dee60-4732-46d4-9143-5d136ac79e69 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:40.612918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2dee60-4732-46d4-9143-5d136ac79e69 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:40.614325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2dee60-4732-46d4-9143-5d136ac79e69 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:40.621396 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07e6e4-3d4a-4533-9874-7d62ff3556ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:40.622466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07e6e4-3d4a-4533-9874-7d62ff3556ca None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:40.623016 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07e6e4-3d4a-4533-9874-7d62ff3556ca None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:40.623513 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07e6e4-3d4a-4533-9874-7d62ff3556ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:40.690075 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 450/899] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 2 08:47:40 2022] POST /identity/v3/services => generated 241 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 08:47:40.704320 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07e6e4-3d4a-4533-9874-7d62ff3556ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:40.706425 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07e6e4-3d4a-4533-9874-7d62ff3556ca None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:40.724863 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68a699-af40-4a40-bc77-5f166282e78b None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:40.724863 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68a699-af40-4a40-bc77-5f166282e78b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:40.725162 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68a699-af40-4a40-bc77-5f166282e78b None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:40.726890 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 451/900] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:47:40.733006 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6881e8a-5177-433a-bf7b-21cda3854cfe None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:40.733006 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6881e8a-5177-433a-bf7b-21cda3854cfe None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:40.733299 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6881e8a-5177-433a-bf7b-21cda3854cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:40.752664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f6881e8a-5177-433a-bf7b-21cda3854cfe None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:40.753930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f6881e8a-5177-433a-bf7b-21cda3854cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:40.788092 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:40.845182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 450/901] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:47:40 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:47:40.845670 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 452/902] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:47:40.847309 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07e6e4-3d4a-4533-9874-7d62ff3556ca None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:40.866753 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6881e8a-5177-433a-bf7b-21cda3854cfe None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:40.971860 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:40.982176 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07e6e4-3d4a-4533-9874-7d62ff3556ca None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:40.992512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6881e8a-5177-433a-bf7b-21cda3854cfe None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:41.025931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f9ca1-04d5-4b3b-8884-4974fb5f7277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:41.026714 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f9ca1-04d5-4b3b-8884-4974fb5f7277 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:41.027011 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f9ca1-04d5-4b3b-8884-4974fb5f7277 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:41.027289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f9ca1-04d5-4b3b-8884-4974fb5f7277 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:41.032508 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46905a98-94b2-4fcb-b690-5938da37a4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:41.033222 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46905a98-94b2-4fcb-b690-5938da37a4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:41.033568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46905a98-94b2-4fcb-b690-5938da37a4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:41.033923 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46905a98-94b2-4fcb-b690-5938da37a4e2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:41.034712 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46905a98-94b2-4fcb-b690-5938da37a4e2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:41.036206 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46905a98-94b2-4fcb-b690-5938da37a4e2 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:41.038745 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f9ca1-04d5-4b3b-8884-4974fb5f7277 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:41.040729 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f9ca1-04d5-4b3b-8884-4974fb5f7277 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:41.046696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-46905a98-94b2-4fcb-b690-5938da37a4e2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 08:47:41.046696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:41.046696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:47:41.046696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:47:41.046696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:47:41.046696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:47:41.046696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:47:41.046696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:41.046696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 08:47:41.046696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:41.046696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:41.046696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:41.046696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:41.046696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:41.046696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:41.046696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:41.048664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:41.048664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:41.048664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:41.048664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:41.048664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:41.048664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:41.048664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:41.048664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:47:41.048664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:47:41.048664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:47:41.048664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:47:41.048664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:41.048664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:41.048664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:41.048664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:41.048664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:41.051017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:41.051017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:41.051017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:41.051017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:41.051017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:41.051017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:41.051017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:41.051017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:41.051017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:41.051017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:41.051017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:41.051017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:47:41.051017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:41.051017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 08:47:41.051017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:41.051017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 453/903] 10.222.0.46 () {56 vars in 1132 bytes} [Tue Aug 2 08:47:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:41.058045 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46905a98-94b2-4fcb-b690-5938da37a4e2 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:41.064302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-516f9ca1-04d5-4b3b-8884-4974fb5f7277 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:41.064302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:41.064302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:41.064302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:41.064302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:41.064302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:41.064302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:41.064302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:41.064302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:41.064302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:41.064302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:41.064302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:41.064302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:41.064302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:41.064302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:41.065522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:41.065522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:41.065522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:41.065522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:41.065522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:41.065522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:41.065522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:41.065522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:41.065522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:41.065522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:41.065522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:41.065522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:41.065522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:41.065522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:41.065522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:41.065522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:41.065522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:41.066820 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:41.066820 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:41.066820 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:41.066820 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:41.066820 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 451/904] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:41.073019 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f9ca1-04d5-4b3b-8884-4974fb5f7277 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:41.093318 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:41.173721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46905a98-94b2-4fcb-b690-5938da37a4e2 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:41.175569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f9ca1-04d5-4b3b-8884-4974fb5f7277 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:41.211602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679c282-2f07-485a-8ced-0c171a067c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:41.212218 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679c282-2f07-485a-8ced-0c171a067c02 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:41.212497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679c282-2f07-485a-8ced-0c171a067c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:41.212771 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679c282-2f07-485a-8ced-0c171a067c02 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:41.213054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f28085-25d9-4964-ba9b-ee42f4d39254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:41.213515 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679c282-2f07-485a-8ced-0c171a067c02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:41.213744 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679c282-2f07-485a-8ced-0c171a067c02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:41.214575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f28085-25d9-4964-ba9b-ee42f4d39254 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:41.215057 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f28085-25d9-4964-ba9b-ee42f4d39254 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:41.215226 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b679c282-2f07-485a-8ced-0c171a067c02 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:41.215686 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f28085-25d9-4964-ba9b-ee42f4d39254 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:41.216663 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f28085-25d9-4964-ba9b-ee42f4d39254 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:41.217201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f28085-25d9-4964-ba9b-ee42f4d39254 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:41.218917 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f28085-25d9-4964-ba9b-ee42f4d39254 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:41.227281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 454/905] 10.222.0.46 () {56 vars in 1126 bytes} [Tue Aug 2 08:47:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:41.229994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 452/906] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:41.232425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f28085-25d9-4964-ba9b-ee42f4d39254 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:41.233940 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679c282-2f07-485a-8ced-0c171a067c02 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:41.250571 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:41.326877 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:41.351476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679c282-2f07-485a-8ced-0c171a067c02 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:41.382634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f28085-25d9-4964-ba9b-ee42f4d39254 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:41.387978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7e80f-6197-4383-a6f6-f5737a4c3df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:41.388622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7e80f-6197-4383-a6f6-f5737a4c3df5 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:41.388904 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7e80f-6197-4383-a6f6-f5737a4c3df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:41.389180 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7e80f-6197-4383-a6f6-f5737a4c3df5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:41.396563 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d7e80f-6197-4383-a6f6-f5737a4c3df5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:41.397845 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d7e80f-6197-4383-a6f6-f5737a4c3df5 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:41.409983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-55d7e80f-6197-4383-a6f6-f5737a4c3df5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:41.409983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:41.409983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:41.409983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:41.409983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:41.409983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:41.409983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:41.409983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:41.409983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:41.409983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:41.409983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:41.409983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:41.409983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:41.409983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:41.409983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:41.410925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:41.410925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:41.410925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:41.410925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:41.410925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:41.410925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:41.410925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:41.410925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:41.410925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:41.410925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:41.410925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:41.410925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:41.410925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:41.410925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:41.410925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:41.410925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:41.410925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:41.411907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:41.411907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:41.411907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:41.411907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:41.411907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:41.411907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:41.411907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:41.411907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:41.416222 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 453/907] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:41.417140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7e80f-6197-4383-a6f6-f5737a4c3df5 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:41.430295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471ecbc-f0ad-428c-8d88-2f858cbc40e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:41.431677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471ecbc-f0ad-428c-8d88-2f858cbc40e0 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:41.432438 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471ecbc-f0ad-428c-8d88-2f858cbc40e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:41.432949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471ecbc-f0ad-428c-8d88-2f858cbc40e0 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:41.434115 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471ecbc-f0ad-428c-8d88-2f858cbc40e0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:41.435122 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471ecbc-f0ad-428c-8d88-2f858cbc40e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:41.436997 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471ecbc-f0ad-428c-8d88-2f858cbc40e0 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:41.445106 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 455/908] 10.222.0.46 () {56 vars in 1126 bytes} [Tue Aug 2 08:47:41 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:41.450902 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471ecbc-f0ad-428c-8d88-2f858cbc40e0 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:41.528966 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:41.547953 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7e80f-6197-4383-a6f6-f5737a4c3df5 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:41.563014 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471ecbc-f0ad-428c-8d88-2f858cbc40e0 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:41.591691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9e5ce-0c15-483c-98c4-470976492e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:41.592636 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9e5ce-0c15-483c-98c4-470976492e40 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:41.592994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9e5ce-0c15-483c-98c4-470976492e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:41.593742 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9e5ce-0c15-483c-98c4-470976492e40 None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:41.594881 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9e5ce-0c15-483c-98c4-470976492e40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:41.595264 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9e5ce-0c15-483c-98c4-470976492e40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:41.597439 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9e5ce-0c15-483c-98c4-470976492e40 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:41.601065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:41.603786 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c738a1-0f55-4914-9d12-f8057750f21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:41.604580 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c738a1-0f55-4914-9d12-f8057750f21f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:41.605008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c738a1-0f55-4914-9d12-f8057750f21f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:41.605430 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c738a1-0f55-4914-9d12-f8057750f21f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:41.606449 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c738a1-0f55-4914-9d12-f8057750f21f None admin] RBAC: Adding query filter params (interface=public, service_id=b863c6220ae641dfa32e8a5b7e6fb3fc, region_id=RegionOne) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:41.606925 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c738a1-0f55-4914-9d12-f8057750f21f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:41.608428 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c738a1-0f55-4914-9d12-f8057750f21f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:41.614313 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 456/910] 10.222.0.46 () {56 vars in 1251 bytes} [Tue Aug 2 08:47:41 2022] GET /identity/v3/endpoints?service_id=b863c6220ae641dfa32e8a5b7e6fb3fc&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:41.617955 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 454/910] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:41.619854 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9e5ce-0c15-483c-98c4-470976492e40 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:41.620971 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c738a1-0f55-4914-9d12-f8057750f21f None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:41.737374 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c738a1-0f55-4914-9d12-f8057750f21f None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:41.746442 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9e5ce-0c15-483c-98c4-470976492e40 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:41.782851 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba0e75-7a9c-4dcb-a0dc-3fe6e715e64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:41.783258 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64fa213-c7da-45c1-b36a-28afcb3b6d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:41.783494 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba0e75-7a9c-4dcb-a0dc-3fe6e715e64d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:41.783778 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba0e75-7a9c-4dcb-a0dc-3fe6e715e64d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:41.784054 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba0e75-7a9c-4dcb-a0dc-3fe6e715e64d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:41.784721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fa213-c7da-45c1-b36a-28afcb3b6d33 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:41.784903 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbba0e75-7a9c-4dcb-a0dc-3fe6e715e64d None admin] RBAC: Adding query filter params (role.id=1606ffdafce14431a59c0f6e8c862c1d, scope.project.id=4227a5b804c0414c99d407bb1415cf49, user.id=ed8114b7a28741268fdff71c866dbc77) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:41.785199 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbba0e75-7a9c-4dcb-a0dc-3fe6e715e64d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:41.785580 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fa213-c7da-45c1-b36a-28afcb3b6d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:41.786078 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64fa213-c7da-45c1-b36a-28afcb3b6d33 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:41.786731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbba0e75-7a9c-4dcb-a0dc-3fe6e715e64d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:41.787186 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64fa213-c7da-45c1-b36a-28afcb3b6d33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:41.788933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64fa213-c7da-45c1-b36a-28afcb3b6d33 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:41.792163 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:41.801927 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 455/911] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:47:41 2022] GET /identity/v3/role_assignments?user.id=ed8114b7a28741268fdff71c866dbc77&scope.project.id=4227a5b804c0414c99d407bb1415cf49&role.id=1606ffdafce14431a59c0f6e8c862c1d => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:41.806528 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 457/912] 10.222.0.46 () {56 vars in 1087 bytes} [Tue Aug 2 08:47:41 2022] GET /identity/v3/services => generated 1497 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 08:47:41.841612 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb567f3b-ddbd-40a8-b606-25eb16644469 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:41.841977 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb567f3b-ddbd-40a8-b606-25eb16644469 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:41.842221 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb567f3b-ddbd-40a8-b606-25eb16644469 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:41.843854 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 456/913] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:41.850506 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1a54e-8139-48bf-b5b0-ef0b109693b6 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:41.851159 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1a54e-8139-48bf-b5b0-ef0b109693b6 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:41.851629 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c1a54e-8139-48bf-b5b0-ef0b109693b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:41.873715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-a3c1a54e-8139-48bf-b5b0-ef0b109693b6 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:41.875349 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a3c1a54e-8139-48bf-b5b0-ef0b109693b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:41.966119 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 458/914] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:47:41.990710 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb567f3b-ddbd-40a8-b606-25eb16644469 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:42.057747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:42.102976 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb567f3b-ddbd-40a8-b606-25eb16644469 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:42.139634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86a6f6-6cd2-410d-a108-23a0b818d7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:42.140278 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86a6f6-6cd2-410d-a108-23a0b818d7fc None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:42.140612 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86a6f6-6cd2-410d-a108-23a0b818d7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:42.140893 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86a6f6-6cd2-410d-a108-23a0b818d7fc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:42.141521 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86a6f6-6cd2-410d-a108-23a0b818d7fc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:42.142913 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86a6f6-6cd2-410d-a108-23a0b818d7fc None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:42.152733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-bf86a6f6-6cd2-410d-a108-23a0b818d7fc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 08:47:42.152733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:42.152733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:47:42.152733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:47:42.152733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:47:42.152733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:47:42.152733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:47:42.152733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:42.152733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 08:47:42.152733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:42.152733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:42.152733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:42.152733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:42.152733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:42.152733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:42.152733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:42.152733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:42.153909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:42.153909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:42.153909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:42.153909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:42.153909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:42.153909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:42.153909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:47:42.153909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:47:42.153909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:47:42.153909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:47:42.153909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:42.153909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:42.153909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:42.153909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:42.153909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:42.153909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:42.154774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:42.154774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:42.154774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:42.154774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:42.154774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:42.154774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:42.154774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:42.154774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:42.154774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:42.154774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:42.154774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:47:42.154774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:42.154774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 08:47:42.154774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:42.154774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 457/915] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 2 08:47:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:42.160028 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c1a54e-8139-48bf-b5b0-ef0b109693b6 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:42.267734 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c1a54e-8139-48bf-b5b0-ef0b109693b6 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:42.299636 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:42.306538 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c3ec0-0219-4402-81e2-87b914307943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:42.307171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c3ec0-0219-4402-81e2-87b914307943 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:42.307461 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c3ec0-0219-4402-81e2-87b914307943 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:42.307729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c3ec0-0219-4402-81e2-87b914307943 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:42.308419 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c3ec0-0219-4402-81e2-87b914307943 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:42.308779 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c3ec0-0219-4402-81e2-87b914307943 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:42.310187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c3ec0-0219-4402-81e2-87b914307943 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:42.315299 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 459/916] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 2 08:47:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:42.320707 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86a6f6-6cd2-410d-a108-23a0b818d7fc None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:42.436246 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86a6f6-6cd2-410d-a108-23a0b818d7fc None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:42.462296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:42.472333 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df69ee-36fb-4583-9890-b3715a233984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:42.473004 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df69ee-36fb-4583-9890-b3715a233984 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:42.473291 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df69ee-36fb-4583-9890-b3715a233984 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:42.473567 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df69ee-36fb-4583-9890-b3715a233984 None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:42.474837 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df69ee-36fb-4583-9890-b3715a233984 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:42.475144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df69ee-36fb-4583-9890-b3715a233984 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:42.476579 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df69ee-36fb-4583-9890-b3715a233984 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:42.481980 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 458/917] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 2 08:47:42 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:42.491273 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c3ec0-0219-4402-81e2-87b914307943 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:42.600091 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c3ec0-0219-4402-81e2-87b914307943 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:42.636178 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4f8a6f-417a-4bbe-940e-3f490ab2c458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:42.636802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f8a6f-417a-4bbe-940e-3f490ab2c458 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:42.637084 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f8a6f-417a-4bbe-940e-3f490ab2c458 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:42.637359 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4f8a6f-417a-4bbe-940e-3f490ab2c458 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:42.638122 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f8a6f-417a-4bbe-940e-3f490ab2c458 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:42.639485 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4f8a6f-417a-4bbe-940e-3f490ab2c458 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:42.729395 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 460/918] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 08:47:42 2022] POST /identity/v3/endpoints => generated 346 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:47:42.843744 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392c9b2-2569-4234-b737-9e356bfb6432 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:42.844121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392c9b2-2569-4234-b737-9e356bfb6432 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:42.844291 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392c9b2-2569-4234-b737-9e356bfb6432 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:42.846108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 459/919] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:42.852065 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd3195-e808-4a56-b09a-4604211038a7 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:42.852327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd3195-e808-4a56-b09a-4604211038a7 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:42.852558 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd3195-e808-4a56-b09a-4604211038a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:42.873040 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-aafd3195-e808-4a56-b09a-4604211038a7 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:42.874282 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-aafd3195-e808-4a56-b09a-4604211038a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:42.927695 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:42.963225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 461/920] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:47:42.987017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392c9b2-2569-4234-b737-9e356bfb6432 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:43.103430 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392c9b2-2569-4234-b737-9e356bfb6432 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:43.145509 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78535e6-23b2-4331-83db-930f7c1db5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:43.146771 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78535e6-23b2-4331-83db-930f7c1db5da None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:43.147054 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78535e6-23b2-4331-83db-930f7c1db5da None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:43.147330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78535e6-23b2-4331-83db-930f7c1db5da None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:43.147953 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78535e6-23b2-4331-83db-930f7c1db5da None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:43.149312 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78535e6-23b2-4331-83db-930f7c1db5da None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:43.158385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-e78535e6-23b2-4331-83db-930f7c1db5da None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:47:43.158385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:43.158385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:47:43.158385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:47:43.158385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:47:43.158385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:47:43.158385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:47:43.158385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:43.158385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:47:43.158385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:43.158385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:43.158385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:43.158385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:43.158385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:43.158385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:43.158385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:43.158385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:43.159424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:43.159424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:43.159424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:43.159424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:43.159424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:43.159424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:43.159424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:47:43.159424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:47:43.159424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:47:43.159424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:47:43.159424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:43.159424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:43.159424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:43.159424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:43.159424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:43.159424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:43.160617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:43.160617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:43.160617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:43.160617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:43.160617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:43.160617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:43.160617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:43.160617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:43.160617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:43.160617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:43.160617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:47:43.160617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:43.160617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:47:43.160617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:43.160617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 460/921] 10.222.0.46 () {56 vars in 1126 bytes} [Tue Aug 2 08:47:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:43.168512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd3195-e808-4a56-b09a-4604211038a7 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:43.274022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd3195-e808-4a56-b09a-4604211038a7 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:43.310366 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073cc8ca-26ba-4d28-93c3-d742175e545e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:43.311329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cc8ca-26ba-4d28-93c3-d742175e545e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:43.311816 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cc8ca-26ba-4d28-93c3-d742175e545e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:43.312312 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cc8ca-26ba-4d28-93c3-d742175e545e None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:43.313198 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073cc8ca-26ba-4d28-93c3-d742175e545e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:43.313742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073cc8ca-26ba-4d28-93c3-d742175e545e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:43.315323 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073cc8ca-26ba-4d28-93c3-d742175e545e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:43.325933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 462/922] 10.222.0.46 () {56 vars in 1122 bytes} [Tue Aug 2 08:47:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:43.327496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78535e6-23b2-4331-83db-930f7c1db5da None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:43.384100 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:43.433452 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78535e6-23b2-4331-83db-930f7c1db5da None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:43.469439 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d6dcf-0622-4c19-91cf-a1e56fa8405e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:43.470556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d6dcf-0622-4c19-91cf-a1e56fa8405e None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:43.471065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d6dcf-0622-4c19-91cf-a1e56fa8405e None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:43.471586 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d6dcf-0622-4c19-91cf-a1e56fa8405e None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:43.472534 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d6dcf-0622-4c19-91cf-a1e56fa8405e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:43.473053 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d6dcf-0622-4c19-91cf-a1e56fa8405e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:43.474747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d6dcf-0622-4c19-91cf-a1e56fa8405e None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:43.483995 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 461/923] 10.222.0.46 () {56 vars in 1122 bytes} [Tue Aug 2 08:47:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:43.490372 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073cc8ca-26ba-4d28-93c3-d742175e545e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:43.537510 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:43.600255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073cc8ca-26ba-4d28-93c3-d742175e545e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:43.636376 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d66e1-96eb-4330-a5fa-6bb118eb2ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:43.637350 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d66e1-96eb-4330-a5fa-6bb118eb2ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:43.637835 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d66e1-96eb-4330-a5fa-6bb118eb2ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:43.638348 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d66e1-96eb-4330-a5fa-6bb118eb2ba0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:43.639319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d66e1-96eb-4330-a5fa-6bb118eb2ba0 None admin] RBAC: Adding query filter params (interface=public, service_id=d509c32233854c2fb1dfce2a6f0e1d3d, region_id=RegionOne) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:43.639832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d66e1-96eb-4330-a5fa-6bb118eb2ba0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:43.641554 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d66e1-96eb-4330-a5fa-6bb118eb2ba0 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:43.651628 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 463/924] 10.222.0.46 () {56 vars in 1251 bytes} [Tue Aug 2 08:47:43 2022] GET /identity/v3/endpoints?service_id=d509c32233854c2fb1dfce2a6f0e1d3d&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:43.653870 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d6dcf-0622-4c19-91cf-a1e56fa8405e None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:43.758381 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d6dcf-0622-4c19-91cf-a1e56fa8405e None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:43.775473 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:43.793652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa0f3d-cb90-4c21-a220-de23468fc217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:43.794308 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa0f3d-cb90-4c21-a220-de23468fc217 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:43.794593 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa0f3d-cb90-4c21-a220-de23468fc217 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:43.814173 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa0f3d-cb90-4c21-a220-de23468fc217 None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:43.814173 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fa0f3d-cb90-4c21-a220-de23468fc217 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:43.814173 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fa0f3d-cb90-4c21-a220-de23468fc217 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:43.814173 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 462/925] 10.222.0.46 () {56 vars in 1087 bytes} [Tue Aug 2 08:47:43 2022] GET /identity/v3/services => generated 1497 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 08:47:43.952196 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26ba0d-9bdb-4637-931c-41047ce7fab7 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:43.952896 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26ba0d-9bdb-4637-931c-41047ce7fab7 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:43.953388 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de26ba0d-9bdb-4637-931c-41047ce7fab7 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:43.955463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 464/926] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:43.962207 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd1ae3-d25d-4cad-a3c1-bc15b9831016 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:43.962861 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd1ae3-d25d-4cad-a3c1-bc15b9831016 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:43.963382 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bd1ae3-d25d-4cad-a3c1-bc15b9831016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:43.983218 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-33bd1ae3-d25d-4cad-a3c1-bc15b9831016 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:43.984507 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-33bd1ae3-d25d-4cad-a3c1-bc15b9831016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:44.032718 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40154110-c595-4925-9dcb-2bfce7b7d051 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:44.033034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40154110-c595-4925-9dcb-2bfce7b7d051 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:44.033281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40154110-c595-4925-9dcb-2bfce7b7d051 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:44.034927 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 465/927] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:44.039843 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bb137-798f-41bb-8caf-f29731505097 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:44.040134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bb137-798f-41bb-8caf-f29731505097 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:44.040418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bb137-798f-41bb-8caf-f29731505097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:44.058653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-7c7bb137-798f-41bb-8caf-f29731505097 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:44.059853 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7c7bb137-798f-41bb-8caf-f29731505097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:44.062446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 463/928] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:47:44.092519 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd1ae3-d25d-4cad-a3c1-bc15b9831016 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:44.132728 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 466/929] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:47:44.155445 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bb137-798f-41bb-8caf-f29731505097 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:44.199785 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:44.204667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bd1ae3-d25d-4cad-a3c1-bc15b9831016 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:44.216865 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:44.241860 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3cfad-b281-43d9-8da1-a8f213357946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:44.243665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3cfad-b281-43d9-8da1-a8f213357946 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:44.244170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3cfad-b281-43d9-8da1-a8f213357946 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:44.244651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f3cfad-b281-43d9-8da1-a8f213357946 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:44.245472 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3cfad-b281-43d9-8da1-a8f213357946 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:44.247264 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f3cfad-b281-43d9-8da1-a8f213357946 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:44.257587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-12f3cfad-b281-43d9-8da1-a8f213357946 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 08:47:44.257587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:44.257587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:47:44.257587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:47:44.257587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:47:44.257587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:47:44.257587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:47:44.257587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:44.257587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 08:47:44.257587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:44.257587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:44.257587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:44.257587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:44.257587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:44.257587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:44.257587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:44.258607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:44.258607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:44.258607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:44.258607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:44.258607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:44.258607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:44.258607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:44.258607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:47:44.258607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:47:44.258607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:47:44.258607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:47:44.258607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:44.258607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:44.258607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:44.258607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:44.258607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:44.259456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:44.259456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:44.259456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:44.259456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:44.259456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:44.259456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:44.259456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:44.259456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:44.259456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:44.259456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:44.259456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:44.259456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:47:44.259456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:44.259456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 08:47:44.259456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:44.262500 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 464/930] 10.222.0.46 () {56 vars in 1132 bytes} [Tue Aug 2 08:47:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:44.268023 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3cfad-b281-43d9-8da1-a8f213357946 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:44.282274 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bb137-798f-41bb-8caf-f29731505097 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:44.323897 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f94415-57a1-4525-b5c8-e26fdb8a1bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:44.324889 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f94415-57a1-4525-b5c8-e26fdb8a1bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:44.325453 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f94415-57a1-4525-b5c8-e26fdb8a1bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:44.326004 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f94415-57a1-4525-b5c8-e26fdb8a1bb7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:44.338463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f94415-57a1-4525-b5c8-e26fdb8a1bb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:47:44.343677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-01f94415-57a1-4525-b5c8-e26fdb8a1bb7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 08:47:44.343677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:44.343677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:44.343677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:44.343677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:44.343677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:44.343677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:44.343677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:44.343677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:44.343677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:44.343677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:44.343677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:44.343677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:47:44.343677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:47:44.343677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:47:44.343677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:47:44.344606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:47:44.344606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:47:44.344606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:47:44.344606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:47:44.344606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:44.344606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:44.344606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:44.344606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:44.344606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:44.344606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:44.344606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:44.344606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:44.344606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:44.344606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:44.344606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:44.344606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:44.345483 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:44.345483 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:44.345483 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:44.345483 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:44.345483 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:47:44.345483 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:47:44.345483 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:47:44.345483 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:47:44.345483 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:47:44.345483 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:47:44.345483 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 08:47:44.345483 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:44.348161 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 467/931] 10.222.0.46 () {56 vars in 1120 bytes} [Tue Aug 2 08:47:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:44.359057 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f94415-57a1-4525-b5c8-e26fdb8a1bb7 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:44.407214 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f3cfad-b281-43d9-8da1-a8f213357946 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:44.475225 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b218a-c7ca-471a-8464-abb59dda74f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:44.475769 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b218a-c7ca-471a-8464-abb59dda74f7 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:44.476047 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b218a-c7ca-471a-8464-abb59dda74f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:44.476318 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b218a-c7ca-471a-8464-abb59dda74f7 None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:44.476996 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b218a-c7ca-471a-8464-abb59dda74f7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:44.477297 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b218a-c7ca-471a-8464-abb59dda74f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:44.479825 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b218a-c7ca-471a-8464-abb59dda74f7 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:44.488832 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 465/932] 10.222.0.46 () {56 vars in 1126 bytes} [Tue Aug 2 08:47:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:44.493981 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b218a-c7ca-471a-8464-abb59dda74f7 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:44.514927 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:44.524444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f94415-57a1-4525-b5c8-e26fdb8a1bb7 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:44.566721 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:44.590408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e65d7-7ac0-43c8-a360-486693e0f398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:44.591045 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e65d7-7ac0-43c8-a360-486693e0f398 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:44.591324 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e65d7-7ac0-43c8-a360-486693e0f398 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:44.591596 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e65d7-7ac0-43c8-a360-486693e0f398 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:44.592326 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e65d7-7ac0-43c8-a360-486693e0f398 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:44.592628 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e65d7-7ac0-43c8-a360-486693e0f398 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:44.594043 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e65d7-7ac0-43c8-a360-486693e0f398 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:44.614156 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 468/933] 10.222.0.46 () {56 vars in 1115 bytes} [Tue Aug 2 08:47:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:44.618948 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e65d7-7ac0-43c8-a360-486693e0f398 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:44.677252 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b218a-c7ca-471a-8464-abb59dda74f7 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:44.732230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e65d7-7ac0-43c8-a360-486693e0f398 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:44.736795 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262bcff-d245-4048-b99f-40ca522ee667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:44.737424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262bcff-d245-4048-b99f-40ca522ee667 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:44.737705 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262bcff-d245-4048-b99f-40ca522ee667 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:44.738002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262bcff-d245-4048-b99f-40ca522ee667 None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:44.738677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262bcff-d245-4048-b99f-40ca522ee667 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:44.738981 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262bcff-d245-4048-b99f-40ca522ee667 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:44.742948 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262bcff-d245-4048-b99f-40ca522ee667 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:44.752555 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 466/934] 10.222.0.46 () {56 vars in 1126 bytes} [Tue Aug 2 08:47:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:44.759268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262bcff-d245-4048-b99f-40ca522ee667 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:44.782436 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235e002-df74-4405-8d78-b594f3f9b246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:44.784598 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235e002-df74-4405-8d78-b594f3f9b246 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:44.785348 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235e002-df74-4405-8d78-b594f3f9b246 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:44.786367 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235e002-df74-4405-8d78-b594f3f9b246 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:44.797954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a235e002-df74-4405-8d78-b594f3f9b246 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:44.799112 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a235e002-df74-4405-8d78-b594f3f9b246 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:44.828560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-a235e002-df74-4405-8d78-b594f3f9b246 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:44.828560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:44.828560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:44.828560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:44.828560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:44.828560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:44.828560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:44.828560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:44.828560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:44.828560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:44.828560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:44.828560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:44.828560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:44.828560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:44.828560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:44.829675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:44.829675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:44.829675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:44.829675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:44.829675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:44.829675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:44.829675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:44.829675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:44.829675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:44.829675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:44.829675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:44.829675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:44.829675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:44.829675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:44.829675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:44.829675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:44.829675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:44.830692 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:44.830692 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:44.830692 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:44.830692 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:44.841502 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 469/935] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:44.844711 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235e002-df74-4405-8d78-b594f3f9b246 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:44.880983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:44.905855 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262bcff-d245-4048-b99f-40ca522ee667 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:44.908436 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:44.969739 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c52c3-0824-4176-99d5-f1c47f5d0137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:44.970642 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c52c3-0824-4176-99d5-f1c47f5d0137 None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:44.971009 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c52c3-0824-4176-99d5-f1c47f5d0137 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:44.971690 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c52c3-0824-4176-99d5-f1c47f5d0137 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:44.972693 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c52c3-0824-4176-99d5-f1c47f5d0137 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:44.974687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c52c3-0824-4176-99d5-f1c47f5d0137 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:45.017986 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235e002-df74-4405-8d78-b594f3f9b246 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:45.051113 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdadfeee-1d7b-4942-8de4-d8743ecde0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:45.051730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdadfeee-1d7b-4942-8de4-d8743ecde0ff None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:45.052040 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdadfeee-1d7b-4942-8de4-d8743ecde0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:45.052319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdadfeee-1d7b-4942-8de4-d8743ecde0ff None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:45.053008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdadfeee-1d7b-4942-8de4-d8743ecde0ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:45.053315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdadfeee-1d7b-4942-8de4-d8743ecde0ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:45.054706 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdadfeee-1d7b-4942-8de4-d8743ecde0ff None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:45.066529 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 470/936] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:45.067524 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad302c-616e-4f09-990d-45ad2d422a68 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:45.068331 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad302c-616e-4f09-990d-45ad2d422a68 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:45.068602 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad302c-616e-4f09-990d-45ad2d422a68 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:45.070017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 471/937] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:45.072712 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad302c-616e-4f09-990d-45ad2d422a68 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:45.090869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 467/938] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 08:47:44 2022] POST /identity/v3/endpoints => generated 347 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:47:45.092524 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750eb36b-b252-4910-9551-e5a17b9d4211 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:45.093040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750eb36b-b252-4910-9551-e5a17b9d4211 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:45.093525 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750eb36b-b252-4910-9551-e5a17b9d4211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:45.113404 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-750eb36b-b252-4910-9551-e5a17b9d4211 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:45.117706 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-750eb36b-b252-4910-9551-e5a17b9d4211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:45.180410 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:45.212149 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 468/939] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:47:45.237400 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad302c-616e-4f09-990d-45ad2d422a68 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:45.238939 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750eb36b-b252-4910-9551-e5a17b9d4211 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:45.291697 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e216b-0e63-427c-91eb-62db0bea872e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:45.292335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e216b-0e63-427c-91eb-62db0bea872e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:45.292566 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e216b-0e63-427c-91eb-62db0bea872e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:45.292841 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e216b-0e63-427c-91eb-62db0bea872e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:45.303884 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e216b-0e63-427c-91eb-62db0bea872e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:45.305623 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e216b-0e63-427c-91eb-62db0bea872e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:45.320460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-a27e216b-0e63-427c-91eb-62db0bea872e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 08:47:45.320460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:45.320460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:45.320460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:45.320460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:45.320460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:45.320460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:45.320460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:45.320460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:45.320460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:45.320460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:45.320460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:45.320460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:45.320460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:45.320460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:45.320460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:45.321482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:45.321482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:45.321482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:45.321482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:45.321482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:45.321482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:45.321482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:45.321482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:45.321482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:45.321482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:45.321482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:45.321482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:45.321482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:45.321482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:45.321482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:45.321482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:45.321482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:45.321482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:45.322515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:45.322515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:45.322515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:45.322515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:45.322515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:45.322515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:45.322515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:45.322515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:45.322515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 08:47:45.322515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:45.322515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 472/940] 10.222.0.46 () {56 vars in 1117 bytes} [Tue Aug 2 08:47:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:45.328121 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e216b-0e63-427c-91eb-62db0bea872e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:45.359831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:45.376700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750eb36b-b252-4910-9551-e5a17b9d4211 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:45.417103 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896de01-7c22-42b3-901e-523f0461f3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:45.417725 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896de01-7c22-42b3-901e-523f0461f3d3 None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:45.418038 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896de01-7c22-42b3-901e-523f0461f3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:45.418314 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896de01-7c22-42b3-901e-523f0461f3d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:45.418989 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5896de01-7c22-42b3-901e-523f0461f3d3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:45.420540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5896de01-7c22-42b3-901e-523f0461f3d3 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:45.423424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: WARNING keystone.server.flask.common [None req-5896de01-7c22-42b3-901e-523f0461f3d3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 02 08:47:45.447700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e216b-0e63-427c-91eb-62db0bea872e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:45.475692 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:45.482079 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.sql.core [None req-5896de01-7c22-42b3-901e-523f0461f3d3 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 02 08:47:45.482079 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 02 08:47:45.482079 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [parameters: {'id': '96ce938fd0ec48d5bebddcf2dfead018', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 02 08:47:45.482079 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80329) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 02 08:47:45.484800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d162a3-e08e-468b-a468-45d722f7abb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:45.485528 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d162a3-e08e-468b-a468-45d722f7abb8 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:45.485864 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d162a3-e08e-468b-a468-45d722f7abb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:45.486168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d162a3-e08e-468b-a468-45d722f7abb8 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:45.486963 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d162a3-e08e-468b-a468-45d722f7abb8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:45.487492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d162a3-e08e-468b-a468-45d722f7abb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:45.489025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d162a3-e08e-468b-a468-45d722f7abb8 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:45.493446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-5896de01-7c22-42b3-901e-523f0461f3d3 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 96ce938fd0ec48d5bebddcf2dfead018.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 96ce938fd0ec48d5bebddcf2dfead018. Aug 02 08:47:45.493446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:45.493446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 08:47:45.493446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 08:47:45.493446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 08:47:45.493446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 08:47:45.493446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 08:47:45.493446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 08:47:45.493446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 08:47:45.493446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application conn.query(q) Aug 02 08:47:45.493446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 08:47:45.493446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 08:47:45.493446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 08:47:45.498331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application result.read() Aug 02 08:47:45.498331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 08:47:45.498331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 08:47:45.498331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 08:47:45.498331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 08:47:45.498331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 08:47:45.498331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 08:47:45.498331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 08:47:45.498331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 08:47:45.498331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 02 08:47:45.498331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:45.498331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 02 08:47:45.498331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:45.498331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:45.498331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 02 08:47:45.498331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 02 08:47:45.498331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 02 08:47:45.498331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 02 08:47:45.499301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 08:47:45.499301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application next(self.gen) Aug 02 08:47:45.499301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 02 08:47:45.499301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application yield resource Aug 02 08:47:45.499301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 08:47:45.499301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application next(self.gen) Aug 02 08:47:45.499301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 02 08:47:45.499301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self.session.rollback() Aug 02 08:47:45.499301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 02 08:47:45.499301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self.force_reraise() Aug 02 08:47:45.499301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 02 08:47:45.499301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise self.value Aug 02 08:47:45.499301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 02 08:47:45.499301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 02 08:47:45.499301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 02 08:47:45.499301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application session.commit() Aug 02 08:47:45.499301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 02 08:47:45.500196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 02 08:47:45.500196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 02 08:47:45.500196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._prepare_impl() Aug 02 08:47:45.500196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 02 08:47:45.500196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self.session.flush() Aug 02 08:47:45.500196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 02 08:47:45.500196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._flush(objects) Aug 02 08:47:45.500196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 02 08:47:45.500196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 02 08:47:45.500196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 02 08:47:45.500196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application compat.raise_( Aug 02 08:47:45.500196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 08:47:45.500196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception Aug 02 08:47:45.500196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 02 08:47:45.500196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application flush_context.execute() Aug 02 08:47:45.500196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 02 08:47:45.500196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rec.execute(self) Aug 02 08:47:45.501093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 02 08:47:45.501093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 02 08:47:45.501093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 02 08:47:45.501093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 02 08:47:45.501093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 02 08:47:45.501093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 02 08:47:45.501093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 02 08:47:45.501093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 02 08:47:45.501093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 02 08:47:45.501093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 02 08:47:45.501093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 02 08:47:45.501093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 02 08:47:45.501093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 02 08:47:45.501093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 02 08:47:45.501093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 02 08:47:45.502347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 02 08:47:45.502347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 08:47:45.502347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception Aug 02 08:47:45.502347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 08:47:45.502347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 08:47:45.502347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 08:47:45.502347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 08:47:45.502347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 08:47:45.502347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 08:47:45.502347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 08:47:45.502347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application conn.query(q) Aug 02 08:47:45.502347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 08:47:45.502347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 08:47:45.502347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 08:47:45.502347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application result.read() Aug 02 08:47:45.502347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 08:47:45.502347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 08:47:45.503503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 08:47:45.503503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 08:47:45.503503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 08:47:45.503503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 08:47:45.503503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 08:47:45.503503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 08:47:45.503503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 02 08:47:45.503503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 02 08:47:45.503503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [parameters: {'id': '96ce938fd0ec48d5bebddcf2dfead018', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 02 08:47:45.503503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 02 08:47:45.503503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:45.503503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:45.503503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:45.503503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:45.503503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 02 08:47:45.504869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 02 08:47:45.504869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 02 08:47:45.504869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 02 08:47:45.504869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 02 08:47:45.504869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:45.504869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:45.504869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:45.504869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:45.504869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:45.504869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:45.504869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:45.504869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:45.504869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:45.504869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:45.504869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:45.504869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:45.504869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:45.515796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:45.515796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 02 08:47:45.515796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 02 08:47:45.515796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:45.515796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:45.515796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 02 08:47:45.515796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 02 08:47:45.515796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 96ce938fd0ec48d5bebddcf2dfead018. Aug 02 08:47:45.515796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:45.515796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 469/941] 10.222.0.46 () {60 vars in 1140 bytes} [Tue Aug 2 08:47:45 2022] POST /identity/v3/projects => generated 262 bytes in 257 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:45.515796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896de01-7c22-42b3-901e-523f0461f3d3 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:45.516515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 473/942] 10.222.0.46 () {56 vars in 1149 bytes} [Tue Aug 2 08:47:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:45.522789 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d162a3-e08e-468b-a468-45d722f7abb8 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:45.639905 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896de01-7c22-42b3-901e-523f0461f3d3 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:45.652080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d162a3-e08e-468b-a468-45d722f7abb8 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:45.685377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f4070f-5db7-4827-a4e5-f2074004c973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:45.686626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4070f-5db7-4827-a4e5-f2074004c973 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:45.687328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4070f-5db7-4827-a4e5-f2074004c973 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:45.687803 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f4070f-5db7-4827-a4e5-f2074004c973 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:45.694330 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3c44a-6211-416d-921e-8e839ed826a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:45.695042 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3c44a-6211-416d-921e-8e839ed826a0 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:45.695388 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3c44a-6211-416d-921e-8e839ed826a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:45.695731 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3c44a-6211-416d-921e-8e839ed826a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:45.697388 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f4070f-5db7-4827-a4e5-f2074004c973 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:45.700186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f4070f-5db7-4827-a4e5-f2074004c973 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:45.707357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad3c44a-6211-416d-921e-8e839ed826a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:45.709036 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad3c44a-6211-416d-921e-8e839ed826a0 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:45.714554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-08f4070f-5db7-4827-a4e5-f2074004c973 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:45.714554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:45.714554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:45.714554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:45.714554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:45.714554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:45.714554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:45.714554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:45.714554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:45.714554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:45.714554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:45.714554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:45.714554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:45.714554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:45.714554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:45.715465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:45.715465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:45.715465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:45.715465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:45.715465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:45.715465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:45.715465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:45.715465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:45.715465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:45.715465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:45.715465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:45.715465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:45.715465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:45.715465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:45.715465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:45.715465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:45.715465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:45.716388 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:45.716388 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:45.716388 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:45.716388 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:45.716388 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:45.716388 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:45.716388 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:45.716388 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:45.723268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:45.723268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 470/943] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:45.724791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f4070f-5db7-4827-a4e5-f2074004c973 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:45.732497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-9ad3c44a-6211-416d-921e-8e839ed826a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:45.732497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:45.732497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:45.732497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:45.732497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:45.732497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:45.732497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:45.732497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:45.732497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:45.732497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:45.732497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:45.732497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:45.732497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:45.732497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:45.732497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:45.739032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:45.739032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:45.739032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:45.739032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:45.739032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:45.739032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:45.739032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:45.739032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:45.739032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:45.739032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:45.739032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:45.739032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:45.739032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:45.739032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:45.739032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:45.739032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:45.739032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:45.740714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:45.740714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:45.740714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:45.740714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:45.740714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 474/944] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:45.744068 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3c44a-6211-416d-921e-8e839ed826a0 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:45.797495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:45.862832 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f4070f-5db7-4827-a4e5-f2074004c973 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:45.882006 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3c44a-6211-416d-921e-8e839ed826a0 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:45.905873 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4126c46-d280-4f69-8d27-1d812d301a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:45.906450 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4126c46-d280-4f69-8d27-1d812d301a92 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:45.906735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4126c46-d280-4f69-8d27-1d812d301a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:45.907011 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4126c46-d280-4f69-8d27-1d812d301a92 None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:45.907988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4126c46-d280-4f69-8d27-1d812d301a92 None admin] RBAC: Adding query filter params (name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:45.908103 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4126c46-d280-4f69-8d27-1d812d301a92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:45.909524 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4126c46-d280-4f69-8d27-1d812d301a92 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:45.925936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553df323-4d9b-4d8a-9b4f-016475b20fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:45.926564 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553df323-4d9b-4d8a-9b4f-016475b20fec None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:45.926844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553df323-4d9b-4d8a-9b4f-016475b20fec None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:45.927649 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553df323-4d9b-4d8a-9b4f-016475b20fec None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:45.928350 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553df323-4d9b-4d8a-9b4f-016475b20fec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:45.928652 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553df323-4d9b-4d8a-9b4f-016475b20fec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:45.930038 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553df323-4d9b-4d8a-9b4f-016475b20fec None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:45.937980 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 471/945] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 2 08:47:45 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:45.937980 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5708d6-a4df-46b1-ac74-e985891a372e None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:45.937980 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5708d6-a4df-46b1-ac74-e985891a372e None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:45.937980 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5708d6-a4df-46b1-ac74-e985891a372e None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:45.937980 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 472/946] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:45.944041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1d0af-5c95-45cb-93b4-9e020c8f118c None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:45.944557 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1d0af-5c95-45cb-93b4-9e020c8f118c None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:45.944992 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d1d0af-5c95-45cb-93b4-9e020c8f118c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:45.954068 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 475/947] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:45.960218 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553df323-4d9b-4d8a-9b4f-016475b20fec None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:45.975111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-44d1d0af-5c95-45cb-93b4-9e020c8f118c None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:45.977579 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-44d1d0af-5c95-45cb-93b4-9e020c8f118c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:46.064953 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:46.077603 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 473/948] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:47:46.095180 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:46.100296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553df323-4d9b-4d8a-9b4f-016475b20fec None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:46.101157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1d0af-5c95-45cb-93b4-9e020c8f118c None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:46.138885 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7446207-1369-4a56-af6b-1ac78bb3a522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:46.139878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7446207-1369-4a56-af6b-1ac78bb3a522 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:46.140388 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7446207-1369-4a56-af6b-1ac78bb3a522 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:46.140921 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7446207-1369-4a56-af6b-1ac78bb3a522 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:46.149154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7446207-1369-4a56-af6b-1ac78bb3a522 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:46.150705 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7446207-1369-4a56-af6b-1ac78bb3a522 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:46.163332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-d7446207-1369-4a56-af6b-1ac78bb3a522 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:46.163332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:46.163332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:46.163332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:46.163332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:46.163332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:46.163332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:46.163332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:46.163332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:46.163332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:46.163332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:46.163332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:46.163332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:46.163332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:46.163332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:46.164277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:46.164277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:46.164277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:46.164277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:46.164277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:46.164277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:46.164277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:46.164277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:46.164277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:46.164277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:46.164277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:46.164277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:46.164277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:46.164277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:46.164277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:46.164277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:46.164277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:46.165342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:46.165342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:46.165342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:46.165342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:46.165342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:46.165342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:46.165342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:46.165342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:46.165342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 476/949] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:46.170443 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7446207-1369-4a56-af6b-1ac78bb3a522 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:46.236170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d1d0af-5c95-45cb-93b4-9e020c8f118c None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:46.274097 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72870e5-d36e-4062-819a-e1a243d01c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:46.274737 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72870e5-d36e-4062-819a-e1a243d01c5b None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:46.275015 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72870e5-d36e-4062-819a-e1a243d01c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:46.275285 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72870e5-d36e-4062-819a-e1a243d01c5b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:46.275932 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72870e5-d36e-4062-819a-e1a243d01c5b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:46.277261 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72870e5-d36e-4062-819a-e1a243d01c5b None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:46.287106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-e72870e5-d36e-4062-819a-e1a243d01c5b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:47:46.287106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:46.287106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:47:46.287106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:47:46.287106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:47:46.287106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:47:46.287106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:47:46.287106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:46.287106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:47:46.287106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:46.287106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:46.287106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:46.287106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:46.287106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:46.287106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:46.287106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:46.287106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:46.288195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:46.288195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:46.288195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:46.288195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:46.288195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:46.288195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:46.288195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:47:46.288195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:47:46.288195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:47:46.288195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:47:46.288195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:46.288195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:46.288195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:46.288195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:46.288195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:46.288195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:46.289055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:46.289055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:46.289055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:46.289055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:46.289055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:46.289055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:46.289055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:46.289055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:46.289055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:46.289055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:46.289055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:47:46.289055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:46.289055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:47:46.289055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:46.293424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 474/950] 10.222.0.46 () {56 vars in 1126 bytes} [Tue Aug 2 08:47:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:46.294982 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72870e5-d36e-4062-819a-e1a243d01c5b None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:46.298282 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7446207-1369-4a56-af6b-1ac78bb3a522 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:46.349868 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1f343-5d10-4368-a9d2-fdee526a5852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:46.351159 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1f343-5d10-4368-a9d2-fdee526a5852 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:46.352061 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1f343-5d10-4368-a9d2-fdee526a5852 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:46.352554 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1f343-5d10-4368-a9d2-fdee526a5852 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:46.353927 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1f343-5d10-4368-a9d2-fdee526a5852 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:46.354465 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1f343-5d10-4368-a9d2-fdee526a5852 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:46.356101 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d1f343-5d10-4368-a9d2-fdee526a5852 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:46.359415 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:46.374349 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 477/951] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:46.381465 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1f343-5d10-4368-a9d2-fdee526a5852 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:46.413428 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:46.414370 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72870e5-d36e-4062-819a-e1a243d01c5b None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:46.452230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a46c64-bd2e-4fb7-9565-2df3a7371733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:46.452897 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a46c64-bd2e-4fb7-9565-2df3a7371733 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:46.453196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a46c64-bd2e-4fb7-9565-2df3a7371733 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:46.453468 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a46c64-bd2e-4fb7-9565-2df3a7371733 None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:46.454325 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a46c64-bd2e-4fb7-9565-2df3a7371733 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:46.454448 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a46c64-bd2e-4fb7-9565-2df3a7371733 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:46.455902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a46c64-bd2e-4fb7-9565-2df3a7371733 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:46.465659 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 475/952] 10.222.0.46 () {56 vars in 1122 bytes} [Tue Aug 2 08:47:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:46.466961 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a46c64-bd2e-4fb7-9565-2df3a7371733 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:46.518623 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1f343-5d10-4368-a9d2-fdee526a5852 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:46.555834 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e090b6-fa04-47f9-a1cb-f83672cbf944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:46.556562 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e090b6-fa04-47f9-a1cb-f83672cbf944 None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:46.556882 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e090b6-fa04-47f9-a1cb-f83672cbf944 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:46.557162 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e090b6-fa04-47f9-a1cb-f83672cbf944 None admin] PATH_INFO: `/v3/projects/4227a5b804c0414c99d407bb1415cf49/users/ed8114b7a28741268fdff71c866dbc77/roles/1606ffdafce14431a59c0f6e8c862c1d` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:46.559526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e090b6-fa04-47f9-a1cb-f83672cbf944 None admin] RBAC: Authorizing `identity:create_grant(project_id=4227a5b804c0414c99d407bb1415cf49, user_id=ed8114b7a28741268fdff71c866dbc77, role_id=1606ffdafce14431a59c0f6e8c862c1d)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:46.561078 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e090b6-fa04-47f9-a1cb-f83672cbf944 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:46.584539 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a46c64-bd2e-4fb7-9565-2df3a7371733 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:46.602792 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:46.620204 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aacd24e-5990-48b5-ab0e-fd6b7a44b47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:46.620823 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aacd24e-5990-48b5-ab0e-fd6b7a44b47d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:46.621196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aacd24e-5990-48b5-ab0e-fd6b7a44b47d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:46.621556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aacd24e-5990-48b5-ab0e-fd6b7a44b47d None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:46.622395 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aacd24e-5990-48b5-ab0e-fd6b7a44b47d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:46.622703 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aacd24e-5990-48b5-ab0e-fd6b7a44b47d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:46.624433 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aacd24e-5990-48b5-ab0e-fd6b7a44b47d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:46.634827 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 476/954] 10.222.0.46 () {56 vars in 1122 bytes} [Tue Aug 2 08:47:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:46.638989 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 478/954] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:47:46 2022] PUT /identity/v3/projects/4227a5b804c0414c99d407bb1415cf49/users/ed8114b7a28741268fdff71c866dbc77/roles/1606ffdafce14431a59c0f6e8c862c1d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:47:46.640107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aacd24e-5990-48b5-ab0e-fd6b7a44b47d None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:46.758131 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aacd24e-5990-48b5-ab0e-fd6b7a44b47d None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:46.798008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce210cd9-4f1a-4d72-aecd-0b81acfa9afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:46.799035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce210cd9-4f1a-4d72-aecd-0b81acfa9afa None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:46.799581 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce210cd9-4f1a-4d72-aecd-0b81acfa9afa None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:46.800066 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce210cd9-4f1a-4d72-aecd-0b81acfa9afa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:46.800995 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce210cd9-4f1a-4d72-aecd-0b81acfa9afa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:46.803132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce210cd9-4f1a-4d72-aecd-0b81acfa9afa None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:46.947825 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 477/955] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 08:47:46 2022] POST /identity/v3/endpoints => generated 349 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:47:47.212413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b6f963-93ec-49ba-bc32-13a4a89f88a5 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:47.212823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b6f963-93ec-49ba-bc32-13a4a89f88a5 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:47.213015 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b6f963-93ec-49ba-bc32-13a4a89f88a5 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:47.214605 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 479/956] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:47.219917 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a23b46-4f29-4931-8bc0-4fb0465defc1 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:47.220215 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a23b46-4f29-4931-8bc0-4fb0465defc1 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:47.220499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a23b46-4f29-4931-8bc0-4fb0465defc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:47.238562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-d0a23b46-4f29-4931-8bc0-4fb0465defc1 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:47.239792 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-d0a23b46-4f29-4931-8bc0-4fb0465defc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:47.274634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:47.318595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 478/957] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:47:47.341296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b6f963-93ec-49ba-bc32-13a4a89f88a5 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:47.424756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:47.448457 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b6f963-93ec-49ba-bc32-13a4a89f88a5 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:47.484383 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84146e9f-8cfd-4917-85b6-e4d968c4a51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:47.484945 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84146e9f-8cfd-4917-85b6-e4d968c4a51d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:47.485224 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84146e9f-8cfd-4917-85b6-e4d968c4a51d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:47.485496 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84146e9f-8cfd-4917-85b6-e4d968c4a51d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:47.486143 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84146e9f-8cfd-4917-85b6-e4d968c4a51d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:47.487497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84146e9f-8cfd-4917-85b6-e4d968c4a51d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:47.496740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-84146e9f-8cfd-4917-85b6-e4d968c4a51d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 08:47:47.496740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:47.496740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:47:47.496740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:47:47.496740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:47:47.496740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:47:47.496740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:47:47.496740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:47.496740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 08:47:47.496740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:47.496740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:47.496740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:47.496740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:47.496740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:47.496740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:47.496740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:47.496740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:47.497909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:47.497909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:47.497909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:47.497909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:47.497909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:47.497909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:47.497909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:47:47.497909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:47:47.497909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:47:47.497909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:47:47.497909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:47.497909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:47.497909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:47.497909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:47.497909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:47.497909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:47.498823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:47.498823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:47.498823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:47.498823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:47.498823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:47.498823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:47.498823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:47.498823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:47.498823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:47.498823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:47.498823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:47:47.498823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:47.498823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 08:47:47.498823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:47.498823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 480/958] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:47.504262 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a23b46-4f29-4931-8bc0-4fb0465defc1 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:47.606671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a23b46-4f29-4931-8bc0-4fb0465defc1 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:47.640352 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7f323-f386-4d7d-94f2-998757709835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:47.640972 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7f323-f386-4d7d-94f2-998757709835 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:47.641256 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7f323-f386-4d7d-94f2-998757709835 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:47.641532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7f323-f386-4d7d-94f2-998757709835 None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:47.642233 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7f323-f386-4d7d-94f2-998757709835 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:47.642540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7f323-f386-4d7d-94f2-998757709835 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:47.643899 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7f323-f386-4d7d-94f2-998757709835 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:47.649494 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 479/959] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 2 08:47:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:47.655172 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84146e9f-8cfd-4917-85b6-e4d968c4a51d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:47.758365 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84146e9f-8cfd-4917-85b6-e4d968c4a51d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:47.792213 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd23a3-39ea-40d9-9ab1-ba25b4b52bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:47.792888 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd23a3-39ea-40d9-9ab1-ba25b4b52bdd None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:47.793171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd23a3-39ea-40d9-9ab1-ba25b4b52bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:47.793445 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd23a3-39ea-40d9-9ab1-ba25b4b52bdd None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:47.794166 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd23a3-39ea-40d9-9ab1-ba25b4b52bdd None admin] RBAC: Adding query filter params (type=compute) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:47.794472 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd23a3-39ea-40d9-9ab1-ba25b4b52bdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:47.795835 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd23a3-39ea-40d9-9ab1-ba25b4b52bdd None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:47.800761 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 481/960] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 2 08:47:47 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:48.135847 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6c382-7e50-4b47-8f16-11a7a7808313 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:48.136197 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6c382-7e50-4b47-8f16-11a7a7808313 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:48.136374 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6c382-7e50-4b47-8f16-11a7a7808313 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:48.137988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 480/961] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:48.143239 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c52d40-0205-4cb4-b82a-bc9947aa67b3 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:48.143539 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c52d40-0205-4cb4-b82a-bc9947aa67b3 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:48.143820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c52d40-0205-4cb4-b82a-bc9947aa67b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:48.158202 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:48.166504 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-23c52d40-0205-4cb4-b82a-bc9947aa67b3 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:48.167732 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-23c52d40-0205-4cb4-b82a-bc9947aa67b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:48.238441 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 482/962] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:47:48.262698 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6c382-7e50-4b47-8f16-11a7a7808313 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:48.326326 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:48.371976 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6c382-7e50-4b47-8f16-11a7a7808313 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:48.412144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8bb41a-154c-4dc7-9a82-9843ad498362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:48.412822 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bb41a-154c-4dc7-9a82-9843ad498362 None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:48.413108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bb41a-154c-4dc7-9a82-9843ad498362 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:48.413382 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8bb41a-154c-4dc7-9a82-9843ad498362 None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:48.414086 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8bb41a-154c-4dc7-9a82-9843ad498362 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:48.415510 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8bb41a-154c-4dc7-9a82-9843ad498362 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:48.472270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.sql.core [None req-df8bb41a-154c-4dc7-9a82-9843ad498362 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 02 08:47:48.472270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 02 08:47:48.472270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [parameters: {'user_id': 'de9f03580e4e4687b7a6fd9fcabe9c10', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 02 08:47:48.472270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80329) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 02 08:47:48.480627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-df8bb41a-154c-4dc7-9a82-9843ad498362 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 02 08:47:48.480627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:48.480627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 08:47:48.480627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 08:47:48.480627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 08:47:48.480627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 08:47:48.480627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 08:47:48.480627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 08:47:48.480627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 08:47:48.480627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application conn.query(q) Aug 02 08:47:48.480627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 08:47:48.480627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 08:47:48.480627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 08:47:48.480627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application result.read() Aug 02 08:47:48.480627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 08:47:48.481506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 08:47:48.481506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 08:47:48.481506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 08:47:48.481506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 08:47:48.481506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 08:47:48.481506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 08:47:48.481506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 08:47:48.481506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 02 08:47:48.481506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:48.481506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 02 08:47:48.481506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:48.481506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:48.481506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 02 08:47:48.481506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 02 08:47:48.481506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 02 08:47:48.481506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 02 08:47:48.481506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 08:47:48.481506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application next(self.gen) Aug 02 08:47:48.482490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 02 08:47:48.482490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application yield resource Aug 02 08:47:48.482490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 08:47:48.482490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application next(self.gen) Aug 02 08:47:48.482490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 02 08:47:48.482490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self.session.rollback() Aug 02 08:47:48.482490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 02 08:47:48.482490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self.force_reraise() Aug 02 08:47:48.482490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 02 08:47:48.482490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise self.value Aug 02 08:47:48.482490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 02 08:47:48.482490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 02 08:47:48.482490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 02 08:47:48.482490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application session.commit() Aug 02 08:47:48.482490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 02 08:47:48.482490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 02 08:47:48.482490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 02 08:47:48.483409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._prepare_impl() Aug 02 08:47:48.483409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 02 08:47:48.483409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self.session.flush() Aug 02 08:47:48.483409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 02 08:47:48.483409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._flush(objects) Aug 02 08:47:48.483409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 02 08:47:48.483409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 02 08:47:48.483409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 02 08:47:48.483409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application compat.raise_( Aug 02 08:47:48.483409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 08:47:48.483409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception Aug 02 08:47:48.483409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 02 08:47:48.483409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application flush_context.execute() Aug 02 08:47:48.483409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 02 08:47:48.483409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rec.execute(self) Aug 02 08:47:48.483409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 02 08:47:48.483409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 02 08:47:48.484328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 02 08:47:48.484328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 02 08:47:48.484328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 02 08:47:48.484328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 02 08:47:48.484328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 02 08:47:48.484328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 02 08:47:48.484328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 02 08:47:48.484328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 02 08:47:48.484328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 02 08:47:48.484328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 02 08:47:48.484328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 02 08:47:48.484328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 02 08:47:48.484328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 02 08:47:48.484328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 02 08:47:48.484328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 08:47:48.484328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception Aug 02 08:47:48.485202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 08:47:48.485202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 08:47:48.485202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 08:47:48.485202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 08:47:48.485202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 08:47:48.485202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 08:47:48.485202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 08:47:48.485202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application conn.query(q) Aug 02 08:47:48.485202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 08:47:48.485202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 08:47:48.485202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 08:47:48.485202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application result.read() Aug 02 08:47:48.485202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 08:47:48.485202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 08:47:48.485202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 08:47:48.485202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 08:47:48.486107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 08:47:48.486107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 08:47:48.486107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 08:47:48.486107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 08:47:48.486107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 02 08:47:48.486107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 02 08:47:48.486107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [parameters: {'user_id': 'de9f03580e4e4687b7a6fd9fcabe9c10', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 02 08:47:48.486107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 02 08:47:48.486107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:48.486107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:48.486107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:48.486107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:48.486107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:48.486107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:48.486107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:48.486931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:48.486931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:48.486931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:48.486931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:48.486931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:48.486931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:48.486931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:48.486931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 02 08:47:48.486931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 02 08:47:48.486931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:48.486931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:48.486931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:48.486931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:48.486931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:48.486931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:48.486931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 02 08:47:48.486931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 02 08:47:48.487852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 02 08:47:48.487852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 02 08:47:48.487852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 02 08:47:48.487852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 02 08:47:48.487852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 02 08:47:48.487852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:48.487852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 481/963] 10.222.0.46 () {60 vars in 1131 bytes} [Tue Aug 2 08:47:48 2022] POST /identity/v3/users => generated 160 bytes in 220 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:48.488256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c52d40-0205-4cb4-b82a-bc9947aa67b3 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:48.596758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c52d40-0205-4cb4-b82a-bc9947aa67b3 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:48.633026 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee6662d-8d63-45b4-8801-e4d4095d166b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:48.633674 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6662d-8d63-45b4-8801-e4d4095d166b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:48.633991 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6662d-8d63-45b4-8801-e4d4095d166b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:48.634269 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee6662d-8d63-45b4-8801-e4d4095d166b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:48.637025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:48.642163 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee6662d-8d63-45b4-8801-e4d4095d166b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:48.643616 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee6662d-8d63-45b4-8801-e4d4095d166b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:48.655549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-0ee6662d-8d63-45b4-8801-e4d4095d166b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 08:47:48.655549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:48.655549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:48.655549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:48.655549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:48.655549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:48.655549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:48.655549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:48.655549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:48.655549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:48.655549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:48.655549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:48.655549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:48.655549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:48.655549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:48.655549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:48.663041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:48.663041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:48.663041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:48.663041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:48.663041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:48.663041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:48.663041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:48.663041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:48.663041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:48.663041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:48.663041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:48.663041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:48.663041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:48.663041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:48.663041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:48.663041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:48.663041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:48.663041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:48.664123 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:48.664123 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:48.664123 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:48.664123 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:48.664123 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:48.664123 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:48.664123 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:48.664123 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:48.664123 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 08:47:48.664123 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:48.664123 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 483/964] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:47:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:48.666882 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8bb41a-154c-4dc7-9a82-9843ad498362 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:48.767844 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:48.778052 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8bb41a-154c-4dc7-9a82-9843ad498362 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:48.813959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca579f4-c371-407a-98a9-b414815f66d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:48.814792 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca579f4-c371-407a-98a9-b414815f66d0 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:48.815288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca579f4-c371-407a-98a9-b414815f66d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:48.815571 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca579f4-c371-407a-98a9-b414815f66d0 None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:48.816342 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca579f4-c371-407a-98a9-b414815f66d0 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:48.816651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca579f4-c371-407a-98a9-b414815f66d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:48.818393 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca579f4-c371-407a-98a9-b414815f66d0 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:48.857068 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b90ad-791b-44e2-86c7-d0170ed111f3 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:48.857708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b90ad-791b-44e2-86c7-d0170ed111f3 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:48.858071 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 482/965] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 2 08:47:48 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:48.858471 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b90ad-791b-44e2-86c7-d0170ed111f3 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:48.862873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 484/966] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:48.875969 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d9b9a-3171-4043-9db7-1afb47d608a5 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:48.876576 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d9b9a-3171-4043-9db7-1afb47d608a5 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:48.877030 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430d9b9a-3171-4043-9db7-1afb47d608a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:48.898353 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-430d9b9a-3171-4043-9db7-1afb47d608a5 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:48.900306 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-430d9b9a-3171-4043-9db7-1afb47d608a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:49.001124 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 483/967] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:47:49.014720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26850bf-7886-4513-8e25-39fd6957d1c3 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:49.015303 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26850bf-7886-4513-8e25-39fd6957d1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:49.015776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26850bf-7886-4513-8e25-39fd6957d1c3 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:49.017644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 485/968] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:49.022579 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d9b9a-3171-4043-9db7-1afb47d608a5 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:49.023976 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ef8b2-b6b8-44f3-9d99-fa530ce00b9b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:49.024491 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ef8b2-b6b8-44f3-9d99-fa530ce00b9b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:49.025049 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ef8b2-b6b8-44f3-9d99-fa530ce00b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:49.045285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-2c6ef8b2-b6b8-44f3-9d99-fa530ce00b9b None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:49.046975 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2c6ef8b2-b6b8-44f3-9d99-fa530ce00b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:49.113733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:49.123387 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 486/969] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:47:49.143600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430d9b9a-3171-4043-9db7-1afb47d608a5 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:49.147137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ef8b2-b6b8-44f3-9d99-fa530ce00b9b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:49.189607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3799b7a-e8cf-4128-af9d-59d531c5b391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:49.190700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3799b7a-e8cf-4128-af9d-59d531c5b391 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:49.190987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3799b7a-e8cf-4128-af9d-59d531c5b391 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:49.191268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3799b7a-e8cf-4128-af9d-59d531c5b391 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:49.204723 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3799b7a-e8cf-4128-af9d-59d531c5b391 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:47:49.209750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-c3799b7a-e8cf-4128-af9d-59d531c5b391 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 08:47:49.209750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:49.209750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:49.209750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:49.209750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:49.209750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:49.209750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:49.209750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:49.209750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:49.209750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:49.209750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:49.209750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:49.209750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:47:49.209750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:47:49.209750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:47:49.209750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:47:49.210941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:47:49.210941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:47:49.210941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:47:49.210941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:47:49.210941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:49.210941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:49.210941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:49.210941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:49.210941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:49.210941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:49.210941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:49.210941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:49.210941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:49.210941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:49.210941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:49.210941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:49.212351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:49.212351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:49.212351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:49.212351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:49.212351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:47:49.212351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:47:49.212351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:47:49.212351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:47:49.212351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:47:49.212351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:47:49.212351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 08:47:49.212351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:49.212351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 484/970] 10.222.0.46 () {56 vars in 1120 bytes} [Tue Aug 2 08:47:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:49.216632 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3799b7a-e8cf-4128-af9d-59d531c5b391 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:49.256747 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ef8b2-b6b8-44f3-9d99-fa530ce00b9b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:49.283221 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:49.295674 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be517247-1626-407b-a2ca-4aa774cedc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:49.296319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be517247-1626-407b-a2ca-4aa774cedc86 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:49.296602 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be517247-1626-407b-a2ca-4aa774cedc86 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:49.297377 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be517247-1626-407b-a2ca-4aa774cedc86 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:49.297492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be517247-1626-407b-a2ca-4aa774cedc86 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:49.299076 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be517247-1626-407b-a2ca-4aa774cedc86 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:49.309219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-be517247-1626-407b-a2ca-4aa774cedc86 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:47:49.309219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:49.309219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:47:49.309219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:47:49.309219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:47:49.309219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:47:49.309219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:47:49.309219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:49.309219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:47:49.309219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:49.309219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:49.309219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:49.309219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:49.309219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:49.309219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:49.309219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:49.309219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:49.310261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:49.310261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:49.310261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:49.310261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:49.310261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:49.310261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:49.310261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:47:49.310261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:47:49.310261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:47:49.310261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:47:49.310261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:49.310261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:49.310261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:49.310261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:49.310261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:49.310261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:49.311124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:49.311124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:49.311124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:49.311124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:49.311124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:49.311124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:49.311124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:49.311124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:49.311124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:49.311124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:49.311124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:47:49.311124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:49.311124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:47:49.311124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:49.311124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 487/971] 10.222.0.46 () {56 vars in 1126 bytes} [Tue Aug 2 08:47:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:49.316912 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be517247-1626-407b-a2ca-4aa774cedc86 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:49.330587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3799b7a-e8cf-4128-af9d-59d531c5b391 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:49.368549 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c1171-cc2d-424f-a3fa-376ec2b18740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:49.369166 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c1171-cc2d-424f-a3fa-376ec2b18740 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:49.369447 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c1171-cc2d-424f-a3fa-376ec2b18740 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:49.369852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c1171-cc2d-424f-a3fa-376ec2b18740 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:49.370681 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c1171-cc2d-424f-a3fa-376ec2b18740 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:49.370990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c1171-cc2d-424f-a3fa-376ec2b18740 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:49.372353 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023c1171-cc2d-424f-a3fa-376ec2b18740 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:49.378146 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:49.382093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 485/972] 10.222.0.46 () {56 vars in 1115 bytes} [Tue Aug 2 08:47:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:49.387585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c1171-cc2d-424f-a3fa-376ec2b18740 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:49.427917 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be517247-1626-407b-a2ca-4aa774cedc86 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:49.465495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648a1be-3801-4f25-b8b7-115a7b7a1b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:49.466067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648a1be-3801-4f25-b8b7-115a7b7a1b72 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:49.466351 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648a1be-3801-4f25-b8b7-115a7b7a1b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:49.466629 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648a1be-3801-4f25-b8b7-115a7b7a1b72 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:49.467306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648a1be-3801-4f25-b8b7-115a7b7a1b72 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:49.467611 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648a1be-3801-4f25-b8b7-115a7b7a1b72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:49.468971 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648a1be-3801-4f25-b8b7-115a7b7a1b72 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:49.474219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 488/973] 10.222.0.46 () {56 vars in 1122 bytes} [Tue Aug 2 08:47:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:49.479925 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648a1be-3801-4f25-b8b7-115a7b7a1b72 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:49.499089 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023c1171-cc2d-424f-a3fa-376ec2b18740 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:49.535697 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2072a-f98d-4636-9d12-46e8938b1ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:49.536850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:49.538549 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2072a-f98d-4636-9d12-46e8938b1ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:49.539130 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2072a-f98d-4636-9d12-46e8938b1ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:49.539580 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c2072a-f98d-4636-9d12-46e8938b1ec5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:49.550931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c2072a-f98d-4636-9d12-46e8938b1ec5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:49.553026 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c2072a-f98d-4636-9d12-46e8938b1ec5 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:49.567133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-76c2072a-f98d-4636-9d12-46e8938b1ec5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:49.567133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:49.567133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:49.567133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:49.567133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:49.567133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:49.567133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:49.567133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:49.567133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:49.567133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:49.567133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:49.567133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:49.567133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:49.567133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:49.567133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:49.568133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:49.568133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:49.568133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:49.568133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:49.568133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:49.568133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:49.568133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:49.568133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:49.568133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:49.568133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:49.568133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:49.568133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:49.568133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:49.568133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:49.568133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:49.568133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:49.568133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:49.569048 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:49.569048 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:49.569048 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:49.569048 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:49.569538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 486/974] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:49.574864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2072a-f98d-4636-9d12-46e8938b1ec5 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:49.588529 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648a1be-3801-4f25-b8b7-115a7b7a1b72 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:49.629608 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:49.632917 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51173fc0-36ef-4566-8108-e6af823a232e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:49.633520 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51173fc0-36ef-4566-8108-e6af823a232e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:49.633798 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51173fc0-36ef-4566-8108-e6af823a232e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:49.634102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51173fc0-36ef-4566-8108-e6af823a232e None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:49.634774 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51173fc0-36ef-4566-8108-e6af823a232e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:49.635080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51173fc0-36ef-4566-8108-e6af823a232e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:49.636520 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51173fc0-36ef-4566-8108-e6af823a232e None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:49.641268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 489/975] 10.222.0.46 () {56 vars in 1122 bytes} [Tue Aug 2 08:47:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:49.647711 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51173fc0-36ef-4566-8108-e6af823a232e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:49.682990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c2072a-f98d-4636-9d12-46e8938b1ec5 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:49.718883 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2b0a2-e745-4999-85ed-f8b983551016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:49.719472 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2b0a2-e745-4999-85ed-f8b983551016 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:49.719756 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2b0a2-e745-4999-85ed-f8b983551016 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:49.720026 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2b0a2-e745-4999-85ed-f8b983551016 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:49.720700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2b0a2-e745-4999-85ed-f8b983551016 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:49.721005 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2b0a2-e745-4999-85ed-f8b983551016 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:49.722414 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2b0a2-e745-4999-85ed-f8b983551016 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:49.735269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 487/976] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:49.741488 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2b0a2-e745-4999-85ed-f8b983551016 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:49.757438 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51173fc0-36ef-4566-8108-e6af823a232e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:49.775039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:49.793356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7d531-9c7c-4fdf-b6ad-533f9a148620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:49.794006 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7d531-9c7c-4fdf-b6ad-533f9a148620 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:49.794289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7d531-9c7c-4fdf-b6ad-533f9a148620 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:49.794560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7d531-9c7c-4fdf-b6ad-533f9a148620 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:49.795322 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7d531-9c7c-4fdf-b6ad-533f9a148620 None admin] RBAC: Adding query filter params (interface=admin, service_id=d509c32233854c2fb1dfce2a6f0e1d3d, region_id=RegionOne) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:49.795633 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7d531-9c7c-4fdf-b6ad-533f9a148620 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:49.796990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7d531-9c7c-4fdf-b6ad-533f9a148620 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:49.807091 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 490/977] 10.222.0.46 () {56 vars in 1249 bytes} [Tue Aug 2 08:47:49 2022] GET /identity/v3/endpoints?service_id=d509c32233854c2fb1dfce2a6f0e1d3d&interface=admin®ion_id=RegionOne => generated 193 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:49.808668 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7d531-9c7c-4fdf-b6ad-533f9a148620 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:49.853792 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2b0a2-e745-4999-85ed-f8b983551016 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:49.867162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:49.888753 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5844e326-791a-4e62-b506-ba53373c361d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:49.889394 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844e326-791a-4e62-b506-ba53373c361d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:49.889684 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844e326-791a-4e62-b506-ba53373c361d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:49.889991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844e326-791a-4e62-b506-ba53373c361d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:49.896987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5844e326-791a-4e62-b506-ba53373c361d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:49.898569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5844e326-791a-4e62-b506-ba53373c361d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:49.908875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7d531-9c7c-4fdf-b6ad-533f9a148620 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:49.910617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-5844e326-791a-4e62-b506-ba53373c361d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 08:47:49.910617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:49.910617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:49.910617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:49.910617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:49.910617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:49.910617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:49.910617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:49.910617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:49.910617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:49.910617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:49.910617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:49.910617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:49.910617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:49.910617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:49.910617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:49.911626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:49.911626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:49.911626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:49.911626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:49.911626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:49.911626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:49.911626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:49.911626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:49.911626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:49.911626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:49.911626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:49.911626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:49.911626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:49.911626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:49.911626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:49.911626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:49.911626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:49.911626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:49.912589 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:49.912589 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:49.912589 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:49.912589 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:49.912589 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:49.912589 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:49.912589 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:49.912589 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:49.912589 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 08:47:49.912589 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:49.912589 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 488/978] 10.222.0.46 () {56 vars in 1117 bytes} [Tue Aug 2 08:47:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:49.913232 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a765832-ecea-42ac-babd-2aea90a36e5f None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:49.913336 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a765832-ecea-42ac-babd-2aea90a36e5f None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:49.913664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a765832-ecea-42ac-babd-2aea90a36e5f None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:49.914879 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 489/979] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:49.917341 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a765832-ecea-42ac-babd-2aea90a36e5f None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:49.946770 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d3df7-d0e0-49f1-ac2e-91e9d11ad773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:49.947392 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d3df7-d0e0-49f1-ac2e-91e9d11ad773 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:49.947678 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d3df7-d0e0-49f1-ac2e-91e9d11ad773 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:49.947958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d3df7-d0e0-49f1-ac2e-91e9d11ad773 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:49.948880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d3df7-d0e0-49f1-ac2e-91e9d11ad773 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:49.950264 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d3df7-d0e0-49f1-ac2e-91e9d11ad773 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:49.956111 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 491/980] 10.222.0.46 () {56 vars in 1087 bytes} [Tue Aug 2 08:47:49 2022] GET /identity/v3/services => generated 1497 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 08:47:49.957251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567b63a-f300-4c96-8820-dc56e4f49830 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:49.957540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567b63a-f300-4c96-8820-dc56e4f49830 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:49.957819 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567b63a-f300-4c96-8820-dc56e4f49830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:49.978827 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-e567b63a-f300-4c96-8820-dc56e4f49830 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:49.979975 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e567b63a-f300-4c96-8820-dc56e4f49830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:50.029928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:50.035597 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a765832-ecea-42ac-babd-2aea90a36e5f None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:50.057488 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 492/981] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:47:50.081806 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567b63a-f300-4c96-8820-dc56e4f49830 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:50.086967 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ea788-3852-4e3f-8498-ce1feadaccef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:50.087600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ea788-3852-4e3f-8498-ce1feadaccef None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:50.087887 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ea788-3852-4e3f-8498-ce1feadaccef None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:50.088174 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ea788-3852-4e3f-8498-ce1feadaccef None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:50.088966 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ea788-3852-4e3f-8498-ce1feadaccef None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:50.089273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ea788-3852-4e3f-8498-ce1feadaccef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:50.090768 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ea788-3852-4e3f-8498-ce1feadaccef None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:50.109130 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 490/982] 10.222.0.46 () {56 vars in 1149 bytes} [Tue Aug 2 08:47:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:50.114926 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ea788-3852-4e3f-8498-ce1feadaccef None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:50.147731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:50.189850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567b63a-f300-4c96-8820-dc56e4f49830 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:50.224573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ea788-3852-4e3f-8498-ce1feadaccef None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:50.236115 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd368881-dcab-4386-8002-9c3b06f375cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:50.236754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd368881-dcab-4386-8002-9c3b06f375cd None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:50.236971 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd368881-dcab-4386-8002-9c3b06f375cd None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:50.237249 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd368881-dcab-4386-8002-9c3b06f375cd None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:50.237983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd368881-dcab-4386-8002-9c3b06f375cd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:50.239479 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd368881-dcab-4386-8002-9c3b06f375cd None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:50.260825 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e269d2-3d55-42c4-aa9b-e207149c0874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:50.261443 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e269d2-3d55-42c4-aa9b-e207149c0874 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:50.261727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e269d2-3d55-42c4-aa9b-e207149c0874 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:50.262040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e269d2-3d55-42c4-aa9b-e207149c0874 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:50.329536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e269d2-3d55-42c4-aa9b-e207149c0874 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:50.331023 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e269d2-3d55-42c4-aa9b-e207149c0874 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:50.341985 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 493/983] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 2 08:47:50 2022] POST /identity/v3/services => generated 242 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 08:47:50.374619 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:50.467992 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 491/984] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:47:50 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 08:47:50.469571 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd368881-dcab-4386-8002-9c3b06f375cd None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:50.510354 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:50.585037 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd368881-dcab-4386-8002-9c3b06f375cd None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:50.619591 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97039620-2b76-4645-b18e-a47698398e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:50.620236 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97039620-2b76-4645-b18e-a47698398e53 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:50.620520 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97039620-2b76-4645-b18e-a47698398e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:50.620797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97039620-2b76-4645-b18e-a47698398e53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:50.631424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97039620-2b76-4645-b18e-a47698398e53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:50.632867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97039620-2b76-4645-b18e-a47698398e53 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:50.647492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-97039620-2b76-4645-b18e-a47698398e53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:50.647492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:50.647492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:50.647492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:50.647492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:50.647492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:50.647492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:50.647492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:50.647492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:50.647492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:50.647492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:50.647492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:50.647492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:50.647492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:50.647492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:50.648418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:50.648418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:50.648418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:50.648418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:50.648418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:50.648418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:50.648418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:50.648418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:50.648418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:50.648418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:50.648418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:50.648418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:50.648418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:50.648418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:50.648418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:50.648418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:50.648418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:50.649492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:50.649492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:50.649492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:50.649492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:50.649492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 494/985] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:50.655711 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e269d2-3d55-42c4-aa9b-e207149c0874 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:50.760520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e269d2-3d55-42c4-aa9b-e207149c0874 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:50.788580 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:50.795311 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23ce3c9-4db9-4961-b728-1b6e60f686a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:50.795996 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ce3c9-4db9-4961-b728-1b6e60f686a2 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:50.796393 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ce3c9-4db9-4961-b728-1b6e60f686a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:50.800458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ce3c9-4db9-4961-b728-1b6e60f686a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:50.800458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23ce3c9-4db9-4961-b728-1b6e60f686a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:50.800458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23ce3c9-4db9-4961-b728-1b6e60f686a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:50.800458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23ce3c9-4db9-4961-b728-1b6e60f686a2 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:50.810953 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 492/986] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:50.816423 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97039620-2b76-4645-b18e-a47698398e53 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:50.904353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:50.920289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97039620-2b76-4645-b18e-a47698398e53 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:50.958649 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45aaab4-e1e0-4f87-baad-b13adf2831ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:50.959594 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45aaab4-e1e0-4f87-baad-b13adf2831ad None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:50.960079 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45aaab4-e1e0-4f87-baad-b13adf2831ad None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:50.960594 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45aaab4-e1e0-4f87-baad-b13adf2831ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:50.967504 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45aaab4-e1e0-4f87-baad-b13adf2831ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:50.969150 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45aaab4-e1e0-4f87-baad-b13adf2831ad None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:50.980993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-c45aaab4-e1e0-4f87-baad-b13adf2831ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:50.980993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:50.980993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:50.980993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:50.980993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:50.980993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:50.980993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:50.980993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:50.980993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:50.980993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:50.980993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:50.980993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:50.980993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:47:50.980993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:47:50.980993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:47:50.981936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:47:50.981936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:50.981936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:50.981936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:50.981936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:50.981936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:50.981936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:50.981936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:50.981936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:50.981936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:50.981936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:50.981936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:50.981936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:50.981936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:50.981936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:50.981936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:50.981936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:50.982867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:47:50.982867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:47:50.982867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:47:50.982867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:47:50.982867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:47:50.982867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:47:50.982867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:47:50.982867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:50.988473 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 495/987] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:50.989236 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23ce3c9-4db9-4961-b728-1b6e60f686a2 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:51.094002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23ce3c9-4db9-4961-b728-1b6e60f686a2 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:51.129036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460cf850-5a7c-4b38-82e7-fe99a49fe134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:51.129661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460cf850-5a7c-4b38-82e7-fe99a49fe134 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:51.129995 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460cf850-5a7c-4b38-82e7-fe99a49fe134 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:51.130275 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460cf850-5a7c-4b38-82e7-fe99a49fe134 None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:51.131018 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460cf850-5a7c-4b38-82e7-fe99a49fe134 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:51.131329 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460cf850-5a7c-4b38-82e7-fe99a49fe134 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:51.132758 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460cf850-5a7c-4b38-82e7-fe99a49fe134 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:51.144461 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 493/988] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:47:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:51.150758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45aaab4-e1e0-4f87-baad-b13adf2831ad None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:51.252620 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45aaab4-e1e0-4f87-baad-b13adf2831ad None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:51.285238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63f4533-109b-4d53-8075-7474aa350acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:51.285851 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63f4533-109b-4d53-8075-7474aa350acc None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:51.286901 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63f4533-109b-4d53-8075-7474aa350acc None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:51.287199 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63f4533-109b-4d53-8075-7474aa350acc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:51.287987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63f4533-109b-4d53-8075-7474aa350acc None admin] RBAC: Adding query filter params (role.id=1606ffdafce14431a59c0f6e8c862c1d, scope.project.id=4227a5b804c0414c99d407bb1415cf49, user.id=ed8114b7a28741268fdff71c866dbc77) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:51.288417 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63f4533-109b-4d53-8075-7474aa350acc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:51.289853 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63f4533-109b-4d53-8075-7474aa350acc None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:51.295765 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 496/989] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:47:51 2022] GET /identity/v3/role_assignments?user.id=ed8114b7a28741268fdff71c866dbc77&scope.project.id=4227a5b804c0414c99d407bb1415cf49&role.id=1606ffdafce14431a59c0f6e8c862c1d => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:51.988105 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c5a0d-098b-4dbc-9ac7-cb2fb4009b30 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:51.988615 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c5a0d-098b-4dbc-9ac7-cb2fb4009b30 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:51.988692 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c5a0d-098b-4dbc-9ac7-cb2fb4009b30 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:51.990621 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 494/990] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:51.995837 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7dee10-a913-44f0-90d9-abeb18c56ce4 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:51.996283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7dee10-a913-44f0-90d9-abeb18c56ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:51.996453 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7dee10-a913-44f0-90d9-abeb18c56ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:52.014836 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-3b7dee10-a913-44f0-90d9-abeb18c56ce4 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:52.016036 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3b7dee10-a913-44f0-90d9-abeb18c56ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:52.018092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:52.083679 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 497/991] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:51 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:47:52.105789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c5a0d-098b-4dbc-9ac7-cb2fb4009b30 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:52.157474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:52.208232 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c5a0d-098b-4dbc-9ac7-cb2fb4009b30 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:52.244798 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808f22f-359d-41a0-a302-1014f6b4cba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:52.245429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808f22f-359d-41a0-a302-1014f6b4cba2 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:52.245708 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808f22f-359d-41a0-a302-1014f6b4cba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:52.246008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5808f22f-359d-41a0-a302-1014f6b4cba2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:52.246629 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5808f22f-359d-41a0-a302-1014f6b4cba2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:52.247982 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5808f22f-359d-41a0-a302-1014f6b4cba2 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:52.257356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-5808f22f-359d-41a0-a302-1014f6b4cba2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:47:52.257356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:52.257356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:47:52.257356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:47:52.257356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:47:52.257356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:47:52.257356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:47:52.257356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:52.257356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:47:52.257356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:52.257356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:52.257356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:52.257356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:52.257356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:52.257356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:52.257356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:52.257356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:52.258595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:52.258595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:52.258595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:52.258595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:52.258595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:52.258595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:52.258595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:47:52.258595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:47:52.258595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:47:52.258595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:47:52.258595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:52.258595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:52.258595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:52.258595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:52.258595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:52.258595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:52.259474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:52.259474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:52.259474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:52.259474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:52.259474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:52.259474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:52.259474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:52.259474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:52.259474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:52.259474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:52.259474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:47:52.259474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:52.259474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 08:47:52.259474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:52.259474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 495/992] 10.222.0.46 () {56 vars in 1126 bytes} [Tue Aug 2 08:47:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:47:52.264518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7dee10-a913-44f0-90d9-abeb18c56ce4 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:52.367438 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7dee10-a913-44f0-90d9-abeb18c56ce4 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:52.402140 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4359c5-4c62-44a5-88c1-17c3676701df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:52.402775 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4359c5-4c62-44a5-88c1-17c3676701df None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:52.403062 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4359c5-4c62-44a5-88c1-17c3676701df None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:52.403342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4359c5-4c62-44a5-88c1-17c3676701df None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:52.404031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4359c5-4c62-44a5-88c1-17c3676701df None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:52.404343 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4359c5-4c62-44a5-88c1-17c3676701df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:52.405739 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4359c5-4c62-44a5-88c1-17c3676701df None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:52.412064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 498/993] 10.222.0.46 () {56 vars in 1122 bytes} [Tue Aug 2 08:47:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:52.418438 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808f22f-359d-41a0-a302-1014f6b4cba2 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:52.478251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd6234-a02f-4c80-bdf9-b8e6ac12fe5c None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:52.478469 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd6234-a02f-4c80-bdf9-b8e6ac12fe5c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:52.478744 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfd6234-a02f-4c80-bdf9-b8e6ac12fe5c None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:52.480348 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 499/994] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:47:52.485253 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e149b19-eb21-4a58-a8ad-fa6189e3a6b9 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:52.485542 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e149b19-eb21-4a58-a8ad-fa6189e3a6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:52.485824 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e149b19-eb21-4a58-a8ad-fa6189e3a6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:52.492655 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:52.504300 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-0e149b19-eb21-4a58-a8ad-fa6189e3a6b9 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:52.511664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0e149b19-eb21-4a58-a8ad-fa6189e3a6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:52.525874 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5808f22f-359d-41a0-a302-1014f6b4cba2 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:52.542091 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:52.565428 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca0396-adbd-4348-b23d-b540366224e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:52.566389 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca0396-adbd-4348-b23d-b540366224e2 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:52.566863 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca0396-adbd-4348-b23d-b540366224e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:52.567284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ca0396-adbd-4348-b23d-b540366224e2 None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:52.567977 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca0396-adbd-4348-b23d-b540366224e2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:52.568428 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca0396-adbd-4348-b23d-b540366224e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:52.570462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ca0396-adbd-4348-b23d-b540366224e2 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:52.573793 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 500/995] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:52 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:47:52.579984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 496/996] 10.222.0.46 () {56 vars in 1122 bytes} [Tue Aug 2 08:47:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:52.585549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e149b19-eb21-4a58-a8ad-fa6189e3a6b9 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:52.598411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca0396-adbd-4348-b23d-b540366224e2 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:52.689249 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e149b19-eb21-4a58-a8ad-fa6189e3a6b9 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:52.708649 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ca0396-adbd-4348-b23d-b540366224e2 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:52.719306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:52.725005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e17a5-4718-4151-b066-e1956d8e39f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:52.725634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e17a5-4718-4151-b066-e1956d8e39f4 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:52.726009 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e17a5-4718-4151-b066-e1956d8e39f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:52.726292 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e17a5-4718-4151-b066-e1956d8e39f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:52.727030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e17a5-4718-4151-b066-e1956d8e39f4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:52.733293 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7e17a5-4718-4151-b066-e1956d8e39f4 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:52.744972 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad201584-da14-4492-8940-fa59a4726d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:52.745611 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad201584-da14-4492-8940-fa59a4726d3c None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:52.745907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad201584-da14-4492-8940-fa59a4726d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:52.746205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad201584-da14-4492-8940-fa59a4726d3c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:52.746827 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad201584-da14-4492-8940-fa59a4726d3c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:52.748181 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad201584-da14-4492-8940-fa59a4726d3c None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:52.757463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-ad201584-da14-4492-8940-fa59a4726d3c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 08:47:52.757463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:52.757463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:47:52.757463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:47:52.757463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:47:52.757463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:47:52.757463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:47:52.757463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:52.757463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 08:47:52.757463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:52.757463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:52.757463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:52.757463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:52.757463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:52.757463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:52.757463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:52.757463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:52.758552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:52.758552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:52.758552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:52.758552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:52.758552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:52.758552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:52.758552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:47:52.758552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:47:52.758552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:47:52.758552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:47:52.758552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:52.758552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:52.758552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:52.758552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:52.758552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:52.758552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:52.759437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:52.759437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:52.759437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:52.759437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:52.759437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:52.759437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:52.759437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:52.759437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:52.759437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:52.759437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:52.759437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:47:52.759437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:52.759437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 08:47:52.759437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:52.759437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 497/997] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:52.766870 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad201584-da14-4492-8940-fa59a4726d3c None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:52.821989 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 501/998] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 08:47:52 2022] POST /identity/v3/endpoints => generated 325 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 08:47:52.822203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:52.881647 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad201584-da14-4492-8940-fa59a4726d3c None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:52.931318 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e60b3ab-102d-481d-a866-109e491f55ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:52.931961 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e60b3ab-102d-481d-a866-109e491f55ec None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:52.932247 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e60b3ab-102d-481d-a866-109e491f55ec None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:52.932527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e60b3ab-102d-481d-a866-109e491f55ec None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:52.933224 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e60b3ab-102d-481d-a866-109e491f55ec None admin] RBAC: Adding query filter params (name=compute) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:52.933537 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e60b3ab-102d-481d-a866-109e491f55ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:52.937441 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e60b3ab-102d-481d-a866-109e491f55ec None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:52.946624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 498/999] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 2 08:47:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:52.948640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e17a5-4718-4151-b066-e1956d8e39f4 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:53.053672 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e17a5-4718-4151-b066-e1956d8e39f4 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:53.089768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93d281-b4bc-489e-9461-937717d4eda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:53.090661 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93d281-b4bc-489e-9461-937717d4eda4 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:53.091095 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93d281-b4bc-489e-9461-937717d4eda4 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:53.091804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93d281-b4bc-489e-9461-937717d4eda4 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:53.092642 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93d281-b4bc-489e-9461-937717d4eda4 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:53.093123 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93d281-b4bc-489e-9461-937717d4eda4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:53.094675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93d281-b4bc-489e-9461-937717d4eda4 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:53.100053 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 502/1000] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 2 08:47:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:53.106268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e60b3ab-102d-481d-a866-109e491f55ec None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:53.213059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e60b3ab-102d-481d-a866-109e491f55ec None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:53.214974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:53.247417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddff43b0-f3be-42f9-9d93-78a638a56c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:53.248499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff43b0-f3be-42f9-9d93-78a638a56c8e None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:53.249025 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff43b0-f3be-42f9-9d93-78a638a56c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:53.249788 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddff43b0-f3be-42f9-9d93-78a638a56c8e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:53.250577 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff43b0-f3be-42f9-9d93-78a638a56c8e None admin] RBAC: Adding query filter params (interface=public, service_id=fa3efa7b36e14a758c8bb0743a1006a9, region_id=RegionOne) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:53.250893 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff43b0-f3be-42f9-9d93-78a638a56c8e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:53.252249 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddff43b0-f3be-42f9-9d93-78a638a56c8e None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:53.264726 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 499/1001] 10.222.0.46 () {56 vars in 1251 bytes} [Tue Aug 2 08:47:53 2022] GET /identity/v3/endpoints?service_id=fa3efa7b36e14a758c8bb0743a1006a9&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:53.266262 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93d281-b4bc-489e-9461-937717d4eda4 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:53.329500 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:53.332548 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab6a3b-4002-49d3-a66d-f8ec684124ab None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:53.333787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab6a3b-4002-49d3-a66d-f8ec684124ab None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:53.333787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fab6a3b-4002-49d3-a66d-f8ec684124ab None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:53.337165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 500/1002] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:47:53.340124 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac58ed5-6bd1-48ba-b697-37a6d3be6446 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:53.340414 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac58ed5-6bd1-48ba-b697-37a6d3be6446 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:53.340694 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac58ed5-6bd1-48ba-b697-37a6d3be6446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:53.359654 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-8ac58ed5-6bd1-48ba-b697-37a6d3be6446 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:53.360835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-8ac58ed5-6bd1-48ba-b697-37a6d3be6446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:53.376610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93d281-b4bc-489e-9461-937717d4eda4 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:53.414158 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb6f72-81b4-4e8a-9f2d-e9ce87f9efad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:53.414781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb6f72-81b4-4e8a-9f2d-e9ce87f9efad None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:53.415087 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb6f72-81b4-4e8a-9f2d-e9ce87f9efad None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:53.415349 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb6f72-81b4-4e8a-9f2d-e9ce87f9efad None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:53.416007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb6f72-81b4-4e8a-9f2d-e9ce87f9efad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:53.417375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb6f72-81b4-4e8a-9f2d-e9ce87f9efad None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:53.424493 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 503/1003] 10.222.0.46 () {56 vars in 1087 bytes} [Tue Aug 2 08:47:53 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 08:47:53.441941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 501/1004] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:53 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:47:53.462327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb6f72-81b4-4e8a-9f2d-e9ce87f9efad None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:53.581780 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb6f72-81b4-4e8a-9f2d-e9ce87f9efad None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:53.604623 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:53.619390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59332668-765d-4970-8413-1b5dd7c6ad24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:53.620036 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59332668-765d-4970-8413-1b5dd7c6ad24 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:53.620318 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59332668-765d-4970-8413-1b5dd7c6ad24 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:53.620586 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59332668-765d-4970-8413-1b5dd7c6ad24 None admin] PATH_INFO: `/v3/services/image` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:53.621210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59332668-765d-4970-8413-1b5dd7c6ad24 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:53.622620 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59332668-765d-4970-8413-1b5dd7c6ad24 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:53.631371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-59332668-765d-4970-8413-1b5dd7c6ad24 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 08:47:53.631371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:53.631371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:47:53.631371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:47:53.631371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:47:53.631371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:47:53.631371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:47:53.631371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:53.631371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 08:47:53.631371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:53.631371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:53.631371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:53.631371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:53.631371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:53.631371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:53.631371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:53.631371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:53.632375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:53.632375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:53.632375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:53.632375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:53.632375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:53.632375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:53.632375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:47:53.632375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:47:53.632375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:47:53.632375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:47:53.632375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:53.632375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:53.632375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:53.632375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:53.632375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:53.632375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:53.633327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:53.633327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:53.633327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:53.633327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:53.633327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:53.633327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:53.633327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:53.633327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:53.633327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:53.633327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:53.633327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:47:53.633327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:53.633327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 08:47:53.633327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:53.633327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 504/1005] 10.222.0.46 () {56 vars in 1105 bytes} [Tue Aug 2 08:47:53 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:53.638401 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac58ed5-6bd1-48ba-b697-37a6d3be6446 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:53.731475 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:53.737927 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac58ed5-6bd1-48ba-b697-37a6d3be6446 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:53.773291 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f85a27f-e8a9-4bec-9d7b-bd761062e442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:53.773981 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85a27f-e8a9-4bec-9d7b-bd761062e442 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:53.774264 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85a27f-e8a9-4bec-9d7b-bd761062e442 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:53.774536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f85a27f-e8a9-4bec-9d7b-bd761062e442 None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:53.775204 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85a27f-e8a9-4bec-9d7b-bd761062e442 None admin] RBAC: Adding query filter params (name=image) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:53.775505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85a27f-e8a9-4bec-9d7b-bd761062e442 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:53.776906 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f85a27f-e8a9-4bec-9d7b-bd761062e442 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:53.786825 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 502/1006] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:53.788014 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59332668-765d-4970-8413-1b5dd7c6ad24 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:53.894645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59332668-765d-4970-8413-1b5dd7c6ad24 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:53.936799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3678db2-d7c7-4833-b803-43c1186f7c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:53.936799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3678db2-d7c7-4833-b803-43c1186f7c85 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:53.936799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3678db2-d7c7-4833-b803-43c1186f7c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:53.936799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3678db2-d7c7-4833-b803-43c1186f7c85 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:53.937227 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3678db2-d7c7-4833-b803-43c1186f7c85 None admin] RBAC: Adding query filter params (type=image) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:53.937227 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3678db2-d7c7-4833-b803-43c1186f7c85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:53.937607 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3678db2-d7c7-4833-b803-43c1186f7c85 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:53.949925 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 505/1007] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:53 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:54.873423 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc49a7-76fb-412c-a504-72e3e519e0b3 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:54.874154 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc49a7-76fb-412c-a504-72e3e519e0b3 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:54.874154 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc49a7-76fb-412c-a504-72e3e519e0b3 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:54.875492 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 503/1008] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:54.880614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded8de5-db58-49a1-b497-1926c24c7382 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:54.881035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded8de5-db58-49a1-b497-1926c24c7382 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:54.881205 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded8de5-db58-49a1-b497-1926c24c7382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:54.900109 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-cded8de5-db58-49a1-b497-1926c24c7382 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:54.901293 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-cded8de5-db58-49a1-b497-1926c24c7382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:54.955225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:54.979477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 506/1009] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:54 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:47:55.002403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc49a7-76fb-412c-a504-72e3e519e0b3 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:55.108924 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc49a7-76fb-412c-a504-72e3e519e0b3 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:55.142230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc579fe1-e5c6-4712-883e-db19507654f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:55.142800 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc579fe1-e5c6-4712-883e-db19507654f5 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:55.143085 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc579fe1-e5c6-4712-883e-db19507654f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:55.143364 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc579fe1-e5c6-4712-883e-db19507654f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:55.144521 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc579fe1-e5c6-4712-883e-db19507654f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:55.146551 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc579fe1-e5c6-4712-883e-db19507654f5 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:55.148317 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 504/1010] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:55.153976 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded8de5-db58-49a1-b497-1926c24c7382 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:55.265563 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded8de5-db58-49a1-b497-1926c24c7382 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:55.302403 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6167369-87ec-404f-86d3-d4a512a07a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:55.303013 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6167369-87ec-404f-86d3-d4a512a07a96 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:55.303289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6167369-87ec-404f-86d3-d4a512a07a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:55.303561 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6167369-87ec-404f-86d3-d4a512a07a96 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:55.304244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6167369-87ec-404f-86d3-d4a512a07a96 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:55.305659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6167369-87ec-404f-86d3-d4a512a07a96 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:55.388448 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 507/1011] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 2 08:47:55 2022] POST /identity/v3/projects => generated 311 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 08:47:55.487143 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6e03e-de7d-4302-ac47-8457f4cd923e None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:55.487431 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6e03e-de7d-4302-ac47-8457f4cd923e None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:55.487887 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6e03e-de7d-4302-ac47-8457f4cd923e None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:55.491195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 505/1012] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:55.496802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cee4e-e6aa-4aee-bc41-05454de1e677 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:55.497105 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cee4e-e6aa-4aee-bc41-05454de1e677 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:55.497389 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cee4e-e6aa-4aee-bc41-05454de1e677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:55.517542 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-521cee4e-e6aa-4aee-bc41-05454de1e677 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:55.518808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-521cee4e-e6aa-4aee-bc41-05454de1e677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:55.548271 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:55.593952 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 508/1013] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:55 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:47:55.620421 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6e03e-de7d-4302-ac47-8457f4cd923e None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:55.649793 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:55.724670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6e03e-de7d-4302-ac47-8457f4cd923e None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:55.758117 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d72133-7dcc-4826-977d-2db124b3644d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:55.758669 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d72133-7dcc-4826-977d-2db124b3644d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:55.758949 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d72133-7dcc-4826-977d-2db124b3644d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:55.759222 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d72133-7dcc-4826-977d-2db124b3644d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:55.759836 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d72133-7dcc-4826-977d-2db124b3644d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:55.761182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d72133-7dcc-4826-977d-2db124b3644d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:55.770001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-d9d72133-7dcc-4826-977d-2db124b3644d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 08:47:55.770001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:55.770001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:47:55.770001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:47:55.770001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:47:55.770001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:47:55.770001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:47:55.770001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:55.770001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 08:47:55.770001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:55.770001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:55.770001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:55.770001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:55.770001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:55.770001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:55.770001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:55.770001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:55.771497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:55.771497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:55.771497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:55.771497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:55.771497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:55.771497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:55.771497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:47:55.771497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:47:55.771497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:47:55.771497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:47:55.771497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:55.771497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:55.771497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:55.771497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:55.771497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:55.771497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:55.772363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:55.772363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:55.772363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:55.772363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:55.772363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:55.772363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:55.772363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:55.772363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:55.772363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:55.772363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:55.772363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:47:55.772363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:55.772363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 08:47:55.772363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:55.772363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 506/1014] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:47:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:55.779697 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cee4e-e6aa-4aee-bc41-05454de1e677 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:55.882511 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cee4e-e6aa-4aee-bc41-05454de1e677 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:55.917622 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec634c4-829b-4ab7-8392-fb64cec2f56f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:55.918282 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec634c4-829b-4ab7-8392-fb64cec2f56f None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:55.918566 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec634c4-829b-4ab7-8392-fb64cec2f56f None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:55.918845 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec634c4-829b-4ab7-8392-fb64cec2f56f None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:55.919527 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec634c4-829b-4ab7-8392-fb64cec2f56f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:55.919836 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec634c4-829b-4ab7-8392-fb64cec2f56f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:55.921211 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec634c4-829b-4ab7-8392-fb64cec2f56f None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:55.930300 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 509/1015] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 2 08:47:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:55.935956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d72133-7dcc-4826-977d-2db124b3644d None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:55.939105 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20d284-f3a8-4c10-851f-5fd05b3cb3a1 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:55.939721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20d284-f3a8-4c10-851f-5fd05b3cb3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:55.939721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b20d284-f3a8-4c10-851f-5fd05b3cb3a1 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:55.940952 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 510/1016] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:55.946120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e3481-5bfb-4121-8bb1-2324b087b8a8 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:55.946387 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e3481-5bfb-4121-8bb1-2324b087b8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:55.946663 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e3481-5bfb-4121-8bb1-2324b087b8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:55.964880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-fa8e3481-5bfb-4121-8bb1-2324b087b8a8 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:55.966175 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-fa8e3481-5bfb-4121-8bb1-2324b087b8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:55.984489 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:56.037996 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 511/1017] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:55 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:47:56.041981 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:56.050748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d72133-7dcc-4826-977d-2db124b3644d None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:56.060559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e3481-5bfb-4121-8bb1-2324b087b8a8 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:56.087615 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f05208-7640-401a-865e-8cbf70f6f8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:56.088326 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f05208-7640-401a-865e-8cbf70f6f8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:56.088683 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f05208-7640-401a-865e-8cbf70f6f8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:56.089045 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f05208-7640-401a-865e-8cbf70f6f8e7 None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:56.089780 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f05208-7640-401a-865e-8cbf70f6f8e7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:56.090187 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f05208-7640-401a-865e-8cbf70f6f8e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:56.091621 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f05208-7640-401a-865e-8cbf70f6f8e7 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:56.096682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 507/1018] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 2 08:47:55 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:56.102155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f05208-7640-401a-865e-8cbf70f6f8e7 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:56.166093 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e3481-5bfb-4121-8bb1-2324b087b8a8 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:56.203808 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f05208-7640-401a-865e-8cbf70f6f8e7 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:56.204145 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257eddce-8850-44cd-95ca-5a7ea34d413c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:56.204835 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257eddce-8850-44cd-95ca-5a7ea34d413c None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:56.205191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257eddce-8850-44cd-95ca-5a7ea34d413c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:56.205474 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257eddce-8850-44cd-95ca-5a7ea34d413c None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:56.206197 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257eddce-8850-44cd-95ca-5a7ea34d413c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:56.207576 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257eddce-8850-44cd-95ca-5a7ea34d413c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:56.239715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21799a-3e37-4db3-9c42-58f2b727b3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:56.240331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21799a-3e37-4db3-9c42-58f2b727b3cd None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:56.240617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21799a-3e37-4db3-9c42-58f2b727b3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:56.240913 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21799a-3e37-4db3-9c42-58f2b727b3cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:56.241593 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21799a-3e37-4db3-9c42-58f2b727b3cd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:56.243020 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21799a-3e37-4db3-9c42-58f2b727b3cd None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:56.295423 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 512/1019] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 2 08:47:56 2022] POST /identity/v3/services => generated 242 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:47:56.319805 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 508/1020] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 08:47:56 2022] POST /identity/v3/endpoints => generated 334 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 08:47:57.445477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c0648-2eb1-4695-a6a7-3d74aafcf299 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:57.446053 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c0648-2eb1-4695-a6a7-3d74aafcf299 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:57.446053 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c0648-2eb1-4695-a6a7-3d74aafcf299 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:57.447579 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 513/1021] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:57.453352 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a519f-ebd3-458a-8a36-d35a5cc8f4e1 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:57.453765 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a519f-ebd3-458a-8a36-d35a5cc8f4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:57.453951 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a519f-ebd3-458a-8a36-d35a5cc8f4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:57.472150 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-1d0a519f-ebd3-458a-8a36-d35a5cc8f4e1 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:57.473332 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-1d0a519f-ebd3-458a-8a36-d35a5cc8f4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:57.482155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:57.549750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 509/1022] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:47:57.575273 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c0648-2eb1-4695-a6a7-3d74aafcf299 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:57.591477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:57.677594 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259c0648-2eb1-4695-a6a7-3d74aafcf299 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:57.711399 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba948ec-a2ac-4014-b61d-f5d1beb907e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:57.711990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba948ec-a2ac-4014-b61d-f5d1beb907e1 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:57.712277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba948ec-a2ac-4014-b61d-f5d1beb907e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:57.712553 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba948ec-a2ac-4014-b61d-f5d1beb907e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:57.713691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba948ec-a2ac-4014-b61d-f5d1beb907e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:57.715144 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba948ec-a2ac-4014-b61d-f5d1beb907e1 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:57.716911 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 514/1023] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:57.722536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a519f-ebd3-458a-8a36-d35a5cc8f4e1 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:57.821731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a519f-ebd3-458a-8a36-d35a5cc8f4e1 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:57.860608 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebbb64b-0059-4103-b100-efbdf1d6c5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:57.861229 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbb64b-0059-4103-b100-efbdf1d6c5f5 None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:57.861511 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbb64b-0059-4103-b100-efbdf1d6c5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:57.861785 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebbb64b-0059-4103-b100-efbdf1d6c5f5 None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:57.862506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebbb64b-0059-4103-b100-efbdf1d6c5f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:57.864010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebbb64b-0059-4103-b100-efbdf1d6c5f5 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:57.874920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:58.003824 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 510/1024] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 2 08:47:57 2022] POST /identity/v3/users => generated 285 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 08:47:58.361304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da6759-1af5-4fb8-b89d-a127da6fc16d None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:58.361647 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da6759-1af5-4fb8-b89d-a127da6fc16d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:58.361835 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da6759-1af5-4fb8-b89d-a127da6fc16d None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:58.363197 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 515/1025] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:47:58.368423 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225211c-311d-42de-99b2-071429a5eb28 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:58.368723 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225211c-311d-42de-99b2-071429a5eb28 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:58.369008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3225211c-311d-42de-99b2-071429a5eb28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:58.388359 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-3225211c-311d-42de-99b2-071429a5eb28 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:58.389561 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-3225211c-311d-42de-99b2-071429a5eb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:58.442596 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc1dbb-a33a-46c4-be97-5f1891042a53 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:58.443118 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc1dbb-a33a-46c4-be97-5f1891042a53 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:58.443409 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc1dbb-a33a-46c4-be97-5f1891042a53 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:58.445189 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 516/1026] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:47:58.450191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d2ae5-c2d9-4081-905f-ca7c4cc9bde5 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:58.450784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d2ae5-c2d9-4081-905f-ca7c4cc9bde5 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:58.450784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5d2ae5-c2d9-4081-905f-ca7c4cc9bde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:58.469936 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 511/1027] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:47:58.470450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-de5d2ae5-c2d9-4081-905f-ca7c4cc9bde5 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:47:58.470450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-de5d2ae5-c2d9-4081-905f-ca7c4cc9bde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:47:58.486464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225211c-311d-42de-99b2-071429a5eb28 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:58.547184 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 517/1028] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:47:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:47:58.571537 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d2ae5-c2d9-4081-905f-ca7c4cc9bde5 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:58.587684 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:58.598090 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3225211c-311d-42de-99b2-071429a5eb28 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:58.626357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:58.639899 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cc98a-4d05-4ac3-968a-7fde2b153ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:58.641032 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cc98a-4d05-4ac3-968a-7fde2b153ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:58.641324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cc98a-4d05-4ac3-968a-7fde2b153ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:58.641612 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5cc98a-4d05-4ac3-968a-7fde2b153ea6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:58.642260 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cc98a-4d05-4ac3-968a-7fde2b153ea6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:58.644085 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5cc98a-4d05-4ac3-968a-7fde2b153ea6 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:58.655651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-1e5cc98a-4d05-4ac3-968a-7fde2b153ea6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 08:47:58.655651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:58.655651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:47:58.655651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:47:58.655651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:47:58.655651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:47:58.655651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:47:58.655651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:58.655651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 08:47:58.655651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:58.655651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:47:58.655651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:58.655651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:58.655651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:58.655651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:58.655651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:58.655651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:58.656664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:58.656664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:58.656664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:58.656664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:58.656664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:58.656664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:58.656664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:47:58.656664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:47:58.656664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:47:58.656664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:47:58.656664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:58.656664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:58.656664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:58.656664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:58.656664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:58.656664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:58.657565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:58.657565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:58.657565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:58.657565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:58.657565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:58.657565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:58.657565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:58.657565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:58.657565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:58.657565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:58.657565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:47:58.657565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:47:58.657565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 08:47:58.657565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:47:58.658418 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 512/1029] 10.222.0.46 () {56 vars in 1105 bytes} [Tue Aug 2 08:47:58 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:58.663804 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cc98a-4d05-4ac3-968a-7fde2b153ea6 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:58.676442 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5d2ae5-c2d9-4081-905f-ca7c4cc9bde5 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:58.712148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb199fd7-2d92-4dba-a410-44fb600c911d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:58.712883 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb199fd7-2d92-4dba-a410-44fb600c911d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:58.713160 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb199fd7-2d92-4dba-a410-44fb600c911d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:58.713681 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb199fd7-2d92-4dba-a410-44fb600c911d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:58.724758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb199fd7-2d92-4dba-a410-44fb600c911d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:47:58.729844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-fb199fd7-2d92-4dba-a410-44fb600c911d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 08:47:58.729844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:58.729844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:58.729844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:58.729844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:58.729844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:58.729844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:58.729844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:58.729844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:58.729844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:58.729844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:58.729844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:58.729844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:47:58.729844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:47:58.729844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:47:58.729844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:47:58.730865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:47:58.730865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:47:58.730865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:47:58.730865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:47:58.730865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:58.730865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:58.730865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:58.730865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:58.730865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:58.730865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:58.730865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:58.730865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:58.730865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:58.730865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:58.730865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:58.730865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:58.731752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:58.731752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:58.731752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:58.731752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:58.731752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:47:58.731752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:47:58.731752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:47:58.731752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:47:58.731752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:47:58.731752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:47:58.731752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 08:47:58.731752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:58.731752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 518/1030] 10.222.0.46 () {56 vars in 1120 bytes} [Tue Aug 2 08:47:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:58.736842 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb199fd7-2d92-4dba-a410-44fb600c911d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:58.778475 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5cc98a-4d05-4ac3-968a-7fde2b153ea6 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:58.811088 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:58.816277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec4585-dd42-48f8-8b24-c96085454bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:58.817105 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec4585-dd42-48f8-8b24-c96085454bfc None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:58.818116 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec4585-dd42-48f8-8b24-c96085454bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:58.818583 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ec4585-dd42-48f8-8b24-c96085454bfc None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:58.819409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ec4585-dd42-48f8-8b24-c96085454bfc None admin] RBAC: Adding query filter params (name=image) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:58.819883 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ec4585-dd42-48f8-8b24-c96085454bfc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:58.821247 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ec4585-dd42-48f8-8b24-c96085454bfc None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:58.826675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 513/1031] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:58.832200 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec4585-dd42-48f8-8b24-c96085454bfc None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:58.841655 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb199fd7-2d92-4dba-a410-44fb600c911d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:58.882889 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05b01e-fdf0-4191-b6bf-db1ceb2e1690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:58.883529 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05b01e-fdf0-4191-b6bf-db1ceb2e1690 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:58.883881 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05b01e-fdf0-4191-b6bf-db1ceb2e1690 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:58.884193 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05b01e-fdf0-4191-b6bf-db1ceb2e1690 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:58.884939 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05b01e-fdf0-4191-b6bf-db1ceb2e1690 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:58.885250 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05b01e-fdf0-4191-b6bf-db1ceb2e1690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:58.886655 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05b01e-fdf0-4191-b6bf-db1ceb2e1690 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:58.891248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:58.895042 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 519/1032] 10.222.0.46 () {56 vars in 1115 bytes} [Tue Aug 2 08:47:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:58.900480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05b01e-fdf0-4191-b6bf-db1ceb2e1690 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:58.951910 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ec4585-dd42-48f8-8b24-c96085454bfc None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:58.985529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d7348-688e-4b6e-b3dc-34d6a2a11523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:58.986181 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d7348-688e-4b6e-b3dc-34d6a2a11523 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:58.986467 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d7348-688e-4b6e-b3dc-34d6a2a11523 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:58.986748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d7348-688e-4b6e-b3dc-34d6a2a11523 None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:58.987421 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d7348-688e-4b6e-b3dc-34d6a2a11523 None admin] RBAC: Adding query filter params (type=image) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:58.987735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d7348-688e-4b6e-b3dc-34d6a2a11523 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:58.989111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5d7348-688e-4b6e-b3dc-34d6a2a11523 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:58.994459 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 514/1033] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:58 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:59.000118 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d7348-688e-4b6e-b3dc-34d6a2a11523 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:59.015805 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05b01e-fdf0-4191-b6bf-db1ceb2e1690 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:59.050475 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dcc025-d37c-461a-9ba7-0ca4fe63a0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:59.051124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dcc025-d37c-461a-9ba7-0ca4fe63a0f6 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:59.051407 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dcc025-d37c-461a-9ba7-0ca4fe63a0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:59.051712 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dcc025-d37c-461a-9ba7-0ca4fe63a0f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:59.053646 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:59.062674 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dcc025-d37c-461a-9ba7-0ca4fe63a0f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:59.064113 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dcc025-d37c-461a-9ba7-0ca4fe63a0f6 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:59.078880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-21dcc025-d37c-461a-9ba7-0ca4fe63a0f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:59.078880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:59.078880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:59.078880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:59.078880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:59.078880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:59.078880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:59.078880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:59.078880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:59.078880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:59.078880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:59.078880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:59.078880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:47:59.078880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:47:59.078880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:47:59.080022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:47:59.080022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:59.080022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:59.080022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:59.080022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:59.080022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:59.080022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:59.080022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:59.080022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:59.080022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:59.080022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:59.080022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:59.080022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:59.080022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:59.080022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:59.080022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:59.080022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:59.081199 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:47:59.081199 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:47:59.081199 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:47:59.081199 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:59.085440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 520/1034] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:59.086335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dcc025-d37c-461a-9ba7-0ca4fe63a0f6 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:59.113602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d7348-688e-4b6e-b3dc-34d6a2a11523 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:59.131125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:59.155091 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6720a13-c8bd-497e-b57c-a2e591d4746e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:59.155639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6720a13-c8bd-497e-b57c-a2e591d4746e None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:59.155924 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6720a13-c8bd-497e-b57c-a2e591d4746e None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:59.156211 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6720a13-c8bd-497e-b57c-a2e591d4746e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:59.156972 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6720a13-c8bd-497e-b57c-a2e591d4746e None admin] RBAC: Adding query filter params (interface=public, service_id=6760129f74074c3087156a521e823d58, region_id=RegionOne) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:59.157287 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6720a13-c8bd-497e-b57c-a2e591d4746e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:59.158704 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6720a13-c8bd-497e-b57c-a2e591d4746e None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:59.164415 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 515/1035] 10.222.0.46 () {56 vars in 1251 bytes} [Tue Aug 2 08:47:58 2022] GET /identity/v3/endpoints?service_id=6760129f74074c3087156a521e823d58&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:59.169764 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6720a13-c8bd-497e-b57c-a2e591d4746e None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:59.201513 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dcc025-d37c-461a-9ba7-0ca4fe63a0f6 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:59.239161 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188243e-0794-495e-8369-678eedda2aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:59.240156 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188243e-0794-495e-8369-678eedda2aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:59.240440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188243e-0794-495e-8369-678eedda2aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:59.240708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188243e-0794-495e-8369-678eedda2aa4 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:59.241402 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e188243e-0794-495e-8369-678eedda2aa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:59.241709 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e188243e-0794-495e-8369-678eedda2aa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:59.243239 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e188243e-0794-495e-8369-678eedda2aa4 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:59.256596 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 521/1036] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:47:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:47:59.262162 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188243e-0794-495e-8369-678eedda2aa4 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:59.274004 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6720a13-c8bd-497e-b57c-a2e591d4746e None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:59.310406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b3b21-7f00-4f55-a418-1f0575434606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:59.311033 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b3b21-7f00-4f55-a418-1f0575434606 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:59.311165 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:59.311313 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b3b21-7f00-4f55-a418-1f0575434606 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:59.311615 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4b3b21-7f00-4f55-a418-1f0575434606 None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:59.312262 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b3b21-7f00-4f55-a418-1f0575434606 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:59.313626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4b3b21-7f00-4f55-a418-1f0575434606 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:59.320985 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 516/1037] 10.222.0.46 () {56 vars in 1087 bytes} [Tue Aug 2 08:47:59 2022] GET /identity/v3/services => generated 1957 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 08:47:59.381501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188243e-0794-495e-8369-678eedda2aa4 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:59.420388 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b4cfb-4856-42ad-ac25-a9bb629608d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:59.421050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b4cfb-4856-42ad-ac25-a9bb629608d8 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:59.421333 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b4cfb-4856-42ad-ac25-a9bb629608d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:59.421609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b4cfb-4856-42ad-ac25-a9bb629608d8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:59.430161 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3b4cfb-4856-42ad-ac25-a9bb629608d8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:59.431641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3b4cfb-4856-42ad-ac25-a9bb629608d8 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:59.445067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-ea3b4cfb-4856-42ad-ac25-a9bb629608d8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:47:59.445067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:47:59.445067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:47:59.445067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:47:59.445067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:47:59.445067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:47:59.445067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:47:59.445067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:47:59.445067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:47:59.445067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:47:59.445067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:47:59.445067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:47:59.445067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:47:59.445067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:47:59.445067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:47:59.445067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:47:59.446142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:47:59.446142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:47:59.446142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:47:59.446142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:59.446142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:47:59.446142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:47:59.446142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:47:59.446142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:47:59.446142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:47:59.446142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:47:59.446142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:47:59.446142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:47:59.446142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:47:59.446142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:47:59.446142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:47:59.446142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:47:59.446142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:47:59.446142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:47:59.447168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:47:59.447168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:47:59.447168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:47:59.447168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:47:59.447168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:47:59.447168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:47:59.447168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:47:59.447168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:47:59.447168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:47:59.447168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:47:59.447168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 522/1038] 10.222.0.46 () {56 vars in 1093 bytes} [Tue Aug 2 08:47:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:47:59.452547 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b3b21-7f00-4f55-a418-1f0575434606 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:59.502976 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:59.557053 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4b3b21-7f00-4f55-a418-1f0575434606 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:59.590490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507f9a5-4803-4996-b367-1e758a455d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:59.591104 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507f9a5-4803-4996-b367-1e758a455d17 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:59.591383 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507f9a5-4803-4996-b367-1e758a455d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:59.591655 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507f9a5-4803-4996-b367-1e758a455d17 None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:59.592444 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f507f9a5-4803-4996-b367-1e758a455d17 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:47:59.592751 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f507f9a5-4803-4996-b367-1e758a455d17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:59.594230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f507f9a5-4803-4996-b367-1e758a455d17 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:59.611965 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 517/1039] 10.222.0.46 () {56 vars in 1133 bytes} [Tue Aug 2 08:47:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:47:59.618031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b4cfb-4856-42ad-ac25-a9bb629608d8 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:47:59.719249 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3b4cfb-4856-42ad-ac25-a9bb629608d8 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:47:59.727282 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:47:59.753642 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1efb2-ea39-4e36-af2b-ed3bc5e6a685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:47:59.754298 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1efb2-ea39-4e36-af2b-ed3bc5e6a685 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:47:59.754576 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1efb2-ea39-4e36-af2b-ed3bc5e6a685 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:47:59.755388 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1efb2-ea39-4e36-af2b-ed3bc5e6a685 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:47:59.824853 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1efb2-ea39-4e36-af2b-ed3bc5e6a685 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:47:59.826339 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1efb2-ea39-4e36-af2b-ed3bc5e6a685 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:47:59.929955 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 523/1040] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:47:59 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:47:59.936276 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507f9a5-4803-4996-b367-1e758a455d17 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:00.036602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507f9a5-4803-4996-b367-1e758a455d17 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:00.038107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:00.071217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7921a789-452f-4f9b-8684-d6753d6e3647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:00.071852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921a789-452f-4f9b-8684-d6753d6e3647 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:00.072132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921a789-452f-4f9b-8684-d6753d6e3647 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:00.072404 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7921a789-452f-4f9b-8684-d6753d6e3647 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:00.082594 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921a789-452f-4f9b-8684-d6753d6e3647 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:00.084627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7921a789-452f-4f9b-8684-d6753d6e3647 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:00.100299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-7921a789-452f-4f9b-8684-d6753d6e3647 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:48:00.100299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:00.100299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:48:00.100299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:48:00.100299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:48:00.100299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:48:00.100299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:48:00.100299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:48:00.100299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:48:00.100299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:48:00.100299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:48:00.100299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:48:00.100299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:48:00.100299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:48:00.100299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:48:00.101203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:48:00.101203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:48:00.101203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:48:00.101203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:48:00.101203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:48:00.101203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:48:00.101203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:48:00.101203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:48:00.101203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:48:00.101203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:48:00.101203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:48:00.101203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:48:00.101203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:48:00.101203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:48:00.101203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:48:00.101203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:48:00.101203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:48:00.106996 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:48:00.106996 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:48:00.106996 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:48:00.106996 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:48:00.106996 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 518/1041] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:47:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:48:00.108988 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1efb2-ea39-4e36-af2b-ed3bc5e6a685 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:00.113779 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87c74d-302c-4ef3-9854-210e26893936 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:00.114177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87c74d-302c-4ef3-9854-210e26893936 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:00.114984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87c74d-302c-4ef3-9854-210e26893936 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:00.116446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 519/1042] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:48:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:48:00.118508 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:00.121160 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f6bf1-0dee-459c-8a28-898e64639218 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:00.121446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f6bf1-0dee-459c-8a28-898e64639218 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:00.121725 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f6bf1-0dee-459c-8a28-898e64639218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:00.139410 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-ba6f6bf1-0dee-459c-8a28-898e64639218 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:48:00.140591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-ba6f6bf1-0dee-459c-8a28-898e64639218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:48:00.209368 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 520/1043] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:48:00 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:48:00.229145 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1efb2-ea39-4e36-af2b-ed3bc5e6a685 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:00.232417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f6bf1-0dee-459c-8a28-898e64639218 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:00.272241 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a5745-faa7-43e9-9b0b-82e9b52cc2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:00.272869 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a5745-faa7-43e9-9b0b-82e9b52cc2cc None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:00.273149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a5745-faa7-43e9-9b0b-82e9b52cc2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:00.273424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a5745-faa7-43e9-9b0b-82e9b52cc2cc None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:00.274504 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a5745-faa7-43e9-9b0b-82e9b52cc2cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:00.274813 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a5745-faa7-43e9-9b0b-82e9b52cc2cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:00.276802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1a5745-faa7-43e9-9b0b-82e9b52cc2cc None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:00.290833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 524/1044] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:00.296706 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a5745-faa7-43e9-9b0b-82e9b52cc2cc None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:00.324432 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:00.340577 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f6bf1-0dee-459c-8a28-898e64639218 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:00.367294 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:00.376582 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b135d8-97ff-49c5-ab5e-23c2cb54c612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:00.377236 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b135d8-97ff-49c5-ab5e-23c2cb54c612 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:00.377529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b135d8-97ff-49c5-ab5e-23c2cb54c612 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:00.377961 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b135d8-97ff-49c5-ab5e-23c2cb54c612 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:00.389526 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b135d8-97ff-49c5-ab5e-23c2cb54c612 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:48:00.394564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-60b135d8-97ff-49c5-ab5e-23c2cb54c612 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:48:00.394564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:00.394564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:48:00.394564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:48:00.394564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:48:00.394564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:48:00.394564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:48:00.394564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:48:00.394564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:48:00.394564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:48:00.394564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:48:00.394564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:48:00.394564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:48:00.394564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:48:00.394564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:48:00.394564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:48:00.396044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:48:00.396044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:48:00.396044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:48:00.396044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:48:00.396044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:48:00.396044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:48:00.396044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:48:00.396044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:48:00.396044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:48:00.396044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:48:00.396044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:48:00.396044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:48:00.396044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:48:00.396044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:48:00.396044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:48:00.396044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:48:00.397010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:48:00.397010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:48:00.397010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:48:00.397010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:48:00.397010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:48:00.397010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:48:00.397010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:48:00.397010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:48:00.397010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:48:00.397010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:48:00.397010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:48:00.397010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:48:00.397010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 521/1045] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:48:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:48:00.402738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b135d8-97ff-49c5-ab5e-23c2cb54c612 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:00.413220 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a5745-faa7-43e9-9b0b-82e9b52cc2cc None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:00.451346 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73a780-bc5f-4f4d-98a3-0a2ddf5afc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:00.451991 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73a780-bc5f-4f4d-98a3-0a2ddf5afc0d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:00.452273 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73a780-bc5f-4f4d-98a3-0a2ddf5afc0d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:00.452548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73a780-bc5f-4f4d-98a3-0a2ddf5afc0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:00.461665 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73a780-bc5f-4f4d-98a3-0a2ddf5afc0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:00.463145 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73a780-bc5f-4f4d-98a3-0a2ddf5afc0d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:00.476347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-3f73a780-bc5f-4f4d-98a3-0a2ddf5afc0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:48:00.476347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:00.476347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:48:00.476347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:48:00.476347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:48:00.476347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:48:00.476347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:48:00.476347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:48:00.476347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:48:00.476347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:48:00.476347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:48:00.476347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:48:00.476347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:48:00.476347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:48:00.476347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:48:00.477315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:48:00.477315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:48:00.477315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:48:00.477315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:48:00.477315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:48:00.477315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:48:00.477315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:48:00.477315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:48:00.477315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:48:00.477315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:48:00.477315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:48:00.477315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:48:00.477315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:48:00.477315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:48:00.477315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:48:00.477315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:48:00.477315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:48:00.478284 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:48:00.478284 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:48:00.478284 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:48:00.478284 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:48:00.478284 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:48:00.478284 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:48:00.478284 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:48:00.478284 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:48:00.478284 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 525/1046] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:48:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:48:00.483153 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73a780-bc5f-4f4d-98a3-0a2ddf5afc0d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:00.524170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b135d8-97ff-49c5-ab5e-23c2cb54c612 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:00.562127 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf901d-e395-4cb0-b27b-29390c164ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:00.562962 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf901d-e395-4cb0-b27b-29390c164ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:00.563243 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf901d-e395-4cb0-b27b-29390c164ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:00.563517 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf901d-e395-4cb0-b27b-29390c164ef6 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:00.564222 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf901d-e395-4cb0-b27b-29390c164ef6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:00.564525 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf901d-e395-4cb0-b27b-29390c164ef6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:00.565875 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf901d-e395-4cb0-b27b-29390c164ef6 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:00.574851 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 522/1047] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:48:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:48:00.580393 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf901d-e395-4cb0-b27b-29390c164ef6 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:00.589861 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:00.590421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73a780-bc5f-4f4d-98a3-0a2ddf5afc0d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:00.617172 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:00.626288 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09b408-3f6c-4516-9c88-5c6ee1b0b867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:00.626890 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09b408-3f6c-4516-9c88-5c6ee1b0b867 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:00.627164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09b408-3f6c-4516-9c88-5c6ee1b0b867 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:00.627442 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09b408-3f6c-4516-9c88-5c6ee1b0b867 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:00.628178 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d09b408-3f6c-4516-9c88-5c6ee1b0b867 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:00.628507 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d09b408-3f6c-4516-9c88-5c6ee1b0b867 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:00.630464 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d09b408-3f6c-4516-9c88-5c6ee1b0b867 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:00.641989 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 526/1048] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:48:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:48:00.648981 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09b408-3f6c-4516-9c88-5c6ee1b0b867 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:00.690525 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf901d-e395-4cb0-b27b-29390c164ef6 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:00.726178 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd9f50-4179-457a-b6ca-583d1f7e7ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:00.726905 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd9f50-4179-457a-b6ca-583d1f7e7ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:00.727257 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd9f50-4179-457a-b6ca-583d1f7e7ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:00.727605 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd9f50-4179-457a-b6ca-583d1f7e7ca5 None admin] PATH_INFO: `/v3/users/6bfcb66257064f45ac3e08cbac49181c` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:00.743747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd9f50-4179-457a-b6ca-583d1f7e7ca5 None admin] RBAC: Authorizing `identity:get_user(user_id=6bfcb66257064f45ac3e08cbac49181c)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:00.745177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd9f50-4179-457a-b6ca-583d1f7e7ca5 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:00.747038 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 523/1049] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:48:00 2022] GET /identity/v3/users/6bfcb66257064f45ac3e08cbac49181c => generated 285 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:48:00.752409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd9f50-4179-457a-b6ca-583d1f7e7ca5 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:00.752768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09b408-3f6c-4516-9c88-5c6ee1b0b867 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:00.792231 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ca72b-424a-4bf4-9d13-f25ac416c167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:00.792853 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ca72b-424a-4bf4-9d13-f25ac416c167 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:00.793134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ca72b-424a-4bf4-9d13-f25ac416c167 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:00.793409 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02ca72b-424a-4bf4-9d13-f25ac416c167 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:00.794233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ca72b-424a-4bf4-9d13-f25ac416c167 None admin] RBAC: Adding query filter params (role.id=1606ffdafce14431a59c0f6e8c862c1d, scope.project.id=4227a5b804c0414c99d407bb1415cf49, user.id=3191d804e8dc4b1a85f490319833ec07) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:00.794546 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ca72b-424a-4bf4-9d13-f25ac416c167 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:00.795984 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02ca72b-424a-4bf4-9d13-f25ac416c167 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:00.814009 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 527/1050] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:48:00 2022] GET /identity/v3/role_assignments?user.id=3191d804e8dc4b1a85f490319833ec07&scope.project.id=4227a5b804c0414c99d407bb1415cf49&role.id=1606ffdafce14431a59c0f6e8c862c1d => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:48:00.837706 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:00.880146 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd9f50-4179-457a-b6ca-583d1f7e7ca5 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:00.917079 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6deec94-0b25-4374-912c-3edba7a583cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:00.917807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6deec94-0b25-4374-912c-3edba7a583cd None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:00.918177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6deec94-0b25-4374-912c-3edba7a583cd None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:00.918516 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6deec94-0b25-4374-912c-3edba7a583cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:01.002158 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6deec94-0b25-4374-912c-3edba7a583cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:01.003767 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6deec94-0b25-4374-912c-3edba7a583cd None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:01.068974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:01.111713 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 524/1051] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:48:00 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 08:48:01.113384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ca72b-424a-4bf4-9d13-f25ac416c167 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:01.166225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:01.216627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02ca72b-424a-4bf4-9d13-f25ac416c167 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:01.250963 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f41905-3842-4395-8198-2c11a5792cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:01.251618 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f41905-3842-4395-8198-2c11a5792cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:01.251898 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f41905-3842-4395-8198-2c11a5792cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:01.252172 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f41905-3842-4395-8198-2c11a5792cd9 None admin] PATH_INFO: `/v3/projects/239afe35723747f6b978e5e6d5aede07` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:01.253357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f41905-3842-4395-8198-2c11a5792cd9 None admin] RBAC: Authorizing `identity:get_project(project_id=239afe35723747f6b978e5e6d5aede07)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:01.254924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f41905-3842-4395-8198-2c11a5792cd9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:01.261688 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 528/1052] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:48:01 2022] GET /identity/v3/projects/239afe35723747f6b978e5e6d5aede07 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:48:01.262955 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6deec94-0b25-4374-912c-3edba7a583cd None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:01.351870 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3c162-f342-4186-91f9-f04a02228686 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:01.352062 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3c162-f342-4186-91f9-f04a02228686 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:01.352913 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f3c162-f342-4186-91f9-f04a02228686 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:01.354452 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 529/1053] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:48:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:48:01.359272 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cff009-f901-4dab-aac4-a5fb57b836ca None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:01.359559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cff009-f901-4dab-aac4-a5fb57b836ca None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:01.359836 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cff009-f901-4dab-aac4-a5fb57b836ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:01.366069 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6deec94-0b25-4374-912c-3edba7a583cd None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:01.378979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-e4cff009-f901-4dab-aac4-a5fb57b836ca None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:48:01.380179 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e4cff009-f901-4dab-aac4-a5fb57b836ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:48:01.408541 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4be5c-58da-494e-9c80-37b241a07711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:01.409157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4be5c-58da-494e-9c80-37b241a07711 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:01.409439 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4be5c-58da-494e-9c80-37b241a07711 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:01.409715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4be5c-58da-494e-9c80-37b241a07711 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:01.411413 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd4be5c-58da-494e-9c80-37b241a07711 None admin] RBAC: Adding query filter params (role.id=8ce0850e15804f3d811e88cc9841e673, scope.project.id=239afe35723747f6b978e5e6d5aede07, user.id=6bfcb66257064f45ac3e08cbac49181c) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:01.411593 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd4be5c-58da-494e-9c80-37b241a07711 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:01.412967 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd4be5c-58da-494e-9c80-37b241a07711 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:01.426136 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 525/1054] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:48:01 2022] GET /identity/v3/role_assignments?user.id=6bfcb66257064f45ac3e08cbac49181c&scope.project.id=239afe35723747f6b978e5e6d5aede07&role.id=8ce0850e15804f3d811e88cc9841e673 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:01.459741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 530/1055] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:48:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:48:01.483410 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4be5c-58da-494e-9c80-37b241a07711 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:01.516132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:01.592980 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4be5c-58da-494e-9c80-37b241a07711 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:01.628033 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4c9e5-01f5-426a-b461-0d36eddd89d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:01.628591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4c9e5-01f5-426a-b461-0d36eddd89d1 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:01.628874 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4c9e5-01f5-426a-b461-0d36eddd89d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:01.629148 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4c9e5-01f5-426a-b461-0d36eddd89d1 None admin] PATH_INFO: `/v3/services/image` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:01.629791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4c9e5-01f5-426a-b461-0d36eddd89d1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:01.631184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4c9e5-01f5-426a-b461-0d36eddd89d1 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:01.639902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-1db4c9e5-01f5-426a-b461-0d36eddd89d1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 08:48:01.639902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:01.639902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:48:01.639902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:48:01.639902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:48:01.639902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:48:01.639902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:48:01.639902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:48:01.639902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 08:48:01.639902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:48:01.639902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:48:01.639902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:48:01.639902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:01.639902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:48:01.639902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:48:01.639902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:48:01.639902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:48:01.640916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:48:01.640916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:48:01.640916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:48:01.640916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:48:01.640916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:48:01.640916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:48:01.640916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:48:01.640916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:48:01.640916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:48:01.640916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:48:01.640916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:48:01.640916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:48:01.640916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:48:01.640916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:48:01.640916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:48:01.640916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:48:01.641796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:48:01.641796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:48:01.641796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:48:01.641796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:48:01.641796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:48:01.641796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:48:01.641796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:48:01.641796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:48:01.641796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:48:01.641796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:48:01.641796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:48:01.641796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:48:01.641796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 08:48:01.641796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:48:01.641796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 526/1056] 10.222.0.46 () {56 vars in 1105 bytes} [Tue Aug 2 08:48:01 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:48:01.649234 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cff009-f901-4dab-aac4-a5fb57b836ca None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:01.699766 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:01.750612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cff009-f901-4dab-aac4-a5fb57b836ca None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:01.784811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e738485-9f87-4374-a521-ee34da3bd350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:01.785437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e738485-9f87-4374-a521-ee34da3bd350 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:01.785720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e738485-9f87-4374-a521-ee34da3bd350 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:01.786577 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e738485-9f87-4374-a521-ee34da3bd350 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:01.787243 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e738485-9f87-4374-a521-ee34da3bd350 None admin] RBAC: Adding query filter params (name=image) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:01.787551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e738485-9f87-4374-a521-ee34da3bd350 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:01.788909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e738485-9f87-4374-a521-ee34da3bd350 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:01.793846 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 531/1057] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:48:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:01.799400 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4c9e5-01f5-426a-b461-0d36eddd89d1 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:01.906186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:01.914647 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4c9e5-01f5-426a-b461-0d36eddd89d1 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:01.958776 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241e0a1-f501-4cac-9a76-c59a11db4014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:01.959396 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241e0a1-f501-4cac-9a76-c59a11db4014 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:01.959674 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241e0a1-f501-4cac-9a76-c59a11db4014 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:01.959946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241e0a1-f501-4cac-9a76-c59a11db4014 None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:01.960616 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1241e0a1-f501-4cac-9a76-c59a11db4014 None admin] RBAC: Adding query filter params (type=image) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:01.960918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1241e0a1-f501-4cac-9a76-c59a11db4014 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:01.962312 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1241e0a1-f501-4cac-9a76-c59a11db4014 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:01.967492 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 527/1058] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:48:01 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:01.973082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e738485-9f87-4374-a521-ee34da3bd350 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:02.071926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e738485-9f87-4374-a521-ee34da3bd350 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:02.085128 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:02.106243 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbf683-224a-416f-98f7-4bda82f333e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:02.106872 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbf683-224a-416f-98f7-4bda82f333e3 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:02.107157 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbf683-224a-416f-98f7-4bda82f333e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:02.107441 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbf683-224a-416f-98f7-4bda82f333e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:02.108163 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbf683-224a-416f-98f7-4bda82f333e3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:02.109523 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbf683-224a-416f-98f7-4bda82f333e3 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:02.185274 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 532/1059] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 08:48:01 2022] POST /identity/v3/endpoints => generated 327 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:48:02.815131 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b37a5-d146-4cf6-a77b-5acb2ce9def3 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:02.815660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b37a5-d146-4cf6-a77b-5acb2ce9def3 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:02.815947 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84b37a5-d146-4cf6-a77b-5acb2ce9def3 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:02.817810 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 528/1060] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:48:02.822820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d82be6-450f-4a18-bbcd-44eb2baabee2 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:02.823244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d82be6-450f-4a18-bbcd-44eb2baabee2 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:02.823666 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d82be6-450f-4a18-bbcd-44eb2baabee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:02.842340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-70d82be6-450f-4a18-bbcd-44eb2baabee2 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:48:02.843526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-70d82be6-450f-4a18-bbcd-44eb2baabee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:48:02.919996 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 533/1061] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:48:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:48:02.943118 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b37a5-d146-4cf6-a77b-5acb2ce9def3 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:03.047820 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84b37a5-d146-4cf6-a77b-5acb2ce9def3 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:03.087296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36865f7a-6c55-4cf0-b9ef-e06d682b874f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:03.087854 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36865f7a-6c55-4cf0-b9ef-e06d682b874f None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:03.088139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36865f7a-6c55-4cf0-b9ef-e06d682b874f None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:03.088415 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36865f7a-6c55-4cf0-b9ef-e06d682b874f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:03.098770 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36865f7a-6c55-4cf0-b9ef-e06d682b874f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:48:03.103667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-36865f7a-6c55-4cf0-b9ef-e06d682b874f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 08:48:03.103667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:03.103667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:48:03.103667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:48:03.103667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:48:03.103667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:48:03.103667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:48:03.103667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:48:03.103667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:48:03.103667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:48:03.103667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:48:03.103667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:48:03.103667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:48:03.103667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:48:03.103667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:48:03.103667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:48:03.104607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:48:03.104607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:48:03.104607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:48:03.104607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:48:03.104607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:48:03.104607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:48:03.104607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:48:03.104607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:48:03.104607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:48:03.104607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:48:03.104607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:48:03.104607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:48:03.104607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:48:03.104607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:48:03.104607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:48:03.104607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:48:03.105477 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:48:03.105477 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:48:03.105477 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:48:03.105477 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:48:03.105477 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:48:03.105477 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:48:03.105477 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:48:03.105477 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:48:03.105477 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:48:03.105477 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:48:03.105477 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 08:48:03.105477 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:48:03.105477 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 529/1062] 10.222.0.46 () {56 vars in 1120 bytes} [Tue Aug 2 08:48:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:48:03.112374 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d82be6-450f-4a18-bbcd-44eb2baabee2 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:03.208423 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:03.215578 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d82be6-450f-4a18-bbcd-44eb2baabee2 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:03.249778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c737d3-943b-4145-9f99-db6e586be935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:03.250430 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c737d3-943b-4145-9f99-db6e586be935 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:03.250715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c737d3-943b-4145-9f99-db6e586be935 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:03.250992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c737d3-943b-4145-9f99-db6e586be935 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:03.251795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c737d3-943b-4145-9f99-db6e586be935 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:03.252112 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c737d3-943b-4145-9f99-db6e586be935 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:03.253485 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c737d3-943b-4145-9f99-db6e586be935 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:03.262859 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 534/1063] 10.222.0.46 () {56 vars in 1115 bytes} [Tue Aug 2 08:48:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:03.268671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36865f7a-6c55-4cf0-b9ef-e06d682b874f None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:03.287472 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:03.374466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36865f7a-6c55-4cf0-b9ef-e06d682b874f None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:03.411265 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b474d6e-b8e2-448b-a475-36566bf02e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:03.411907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b474d6e-b8e2-448b-a475-36566bf02e41 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:03.412196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b474d6e-b8e2-448b-a475-36566bf02e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:03.412474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b474d6e-b8e2-448b-a475-36566bf02e41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:03.414809 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067ab72-1cac-4a8d-9ea1-63f2a6edc653 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:03.415102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067ab72-1cac-4a8d-9ea1-63f2a6edc653 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:03.415381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067ab72-1cac-4a8d-9ea1-63f2a6edc653 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:03.416856 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 535/1064] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:48:03.421718 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c125d2-d23e-4fbd-b572-f6fafc96d997 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:03.422038 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c125d2-d23e-4fbd-b572-f6fafc96d997 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:03.422320 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c125d2-d23e-4fbd-b572-f6fafc96d997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:03.424254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b474d6e-b8e2-448b-a475-36566bf02e41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:03.425688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b474d6e-b8e2-448b-a475-36566bf02e41 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:03.440655 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-a7c125d2-d23e-4fbd-b572-f6fafc96d997 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:48:03.440908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-0b474d6e-b8e2-448b-a475-36566bf02e41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:48:03.440908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:03.440908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:48:03.440908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:48:03.440908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:48:03.440908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:48:03.440908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:48:03.440908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:48:03.440908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:48:03.440908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:48:03.440908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:48:03.440908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:48:03.440908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:48:03.440908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:48:03.440908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:48:03.441787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:48:03.441787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:48:03.441787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:48:03.441787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:48:03.441787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:48:03.441787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:48:03.441787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:48:03.441787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:48:03.441787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:48:03.441787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:48:03.441787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:48:03.441787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:48:03.441787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:48:03.441787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:48:03.441787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:48:03.441787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:48:03.441787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:48:03.448097 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:48:03.448097 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:48:03.448097 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:48:03.448097 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:48:03.448589 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a7c125d2-d23e-4fbd-b572-f6fafc96d997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:48:03.448676 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 530/1065] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:48:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:48:03.448676 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b474d6e-b8e2-448b-a475-36566bf02e41 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:03.516819 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 536/1066] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:48:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:48:03.519158 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:03.539562 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c125d2-d23e-4fbd-b572-f6fafc96d997 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:03.555200 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b474d6e-b8e2-448b-a475-36566bf02e41 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:03.590297 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b379149-7889-4c6e-9781-8b5318576e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:03.590917 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b379149-7889-4c6e-9781-8b5318576e54 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:03.591208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b379149-7889-4c6e-9781-8b5318576e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:03.591489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b379149-7889-4c6e-9781-8b5318576e54 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:03.592182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b379149-7889-4c6e-9781-8b5318576e54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:03.592493 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b379149-7889-4c6e-9781-8b5318576e54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:03.593857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b379149-7889-4c6e-9781-8b5318576e54 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:03.610806 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 531/1067] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:03.611799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b379149-7889-4c6e-9781-8b5318576e54 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:03.629442 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:03.646005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c125d2-d23e-4fbd-b572-f6fafc96d997 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:03.681405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4de1ca-f8ca-4fff-af51-4f536e97f6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:03.682086 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4de1ca-f8ca-4fff-af51-4f536e97f6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:03.682373 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4de1ca-f8ca-4fff-af51-4f536e97f6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:03.682651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4de1ca-f8ca-4fff-af51-4f536e97f6d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:03.694705 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4de1ca-f8ca-4fff-af51-4f536e97f6d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:48:03.699957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-4c4de1ca-f8ca-4fff-af51-4f536e97f6d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:48:03.699957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:03.699957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:48:03.699957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:48:03.699957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:48:03.699957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:48:03.699957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:48:03.699957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:48:03.699957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:48:03.699957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:48:03.699957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:48:03.699957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:48:03.699957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:48:03.699957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:48:03.699957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:48:03.699957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:48:03.700895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:48:03.700895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:48:03.700895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:48:03.700895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:48:03.700895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:48:03.700895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:48:03.700895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:48:03.700895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:48:03.700895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:48:03.700895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:48:03.700895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:48:03.700895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:48:03.700895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:48:03.700895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:48:03.700895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:48:03.700895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:48:03.702020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:48:03.702020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:48:03.702020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:48:03.702020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:48:03.702020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:48:03.702020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:48:03.702020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:48:03.702020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:48:03.702020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:48:03.702020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:48:03.702020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:48:03.702020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:48:03.702020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 537/1068] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:48:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:48:03.707021 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4de1ca-f8ca-4fff-af51-4f536e97f6d7 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:03.722783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b379149-7889-4c6e-9781-8b5318576e54 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:03.751433 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:03.759428 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20de57-1ede-4514-867b-7918fa1f3a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:03.763156 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20de57-1ede-4514-867b-7918fa1f3a17 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:03.763156 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20de57-1ede-4514-867b-7918fa1f3a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:03.763156 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20de57-1ede-4514-867b-7918fa1f3a17 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:03.767731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e20de57-1ede-4514-867b-7918fa1f3a17 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:03.769162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e20de57-1ede-4514-867b-7918fa1f3a17 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:03.782083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-5e20de57-1ede-4514-867b-7918fa1f3a17 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:48:03.782083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:03.782083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:48:03.782083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:48:03.782083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:48:03.782083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:48:03.782083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:48:03.782083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:48:03.782083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:48:03.782083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:48:03.782083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:48:03.782083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:48:03.782083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:48:03.782083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:48:03.782083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:48:03.782083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:48:03.783082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:48:03.783082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:48:03.783082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:48:03.783082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:48:03.783082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:48:03.783082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:48:03.783082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:48:03.783082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:48:03.783082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:48:03.783082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:48:03.783082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:48:03.783082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:48:03.783082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:48:03.783082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:48:03.783082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:48:03.783082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:48:03.783082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:48:03.783082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:48:03.784051 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:48:03.784051 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:48:03.784051 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:48:03.784051 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:48:03.784051 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:48:03.784051 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:48:03.784051 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:48:03.784051 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:48:03.784051 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:48:03.784051 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:48:03.791021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 532/1069] 10.222.0.46 () {56 vars in 1093 bytes} [Tue Aug 2 08:48:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:48:03.793450 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20de57-1ede-4514-867b-7918fa1f3a17 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:03.820576 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4de1ca-f8ca-4fff-af51-4f536e97f6d7 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:03.859414 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8233d23c-1d32-475d-b0c2-5e5330f1395d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:03.860353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233d23c-1d32-475d-b0c2-5e5330f1395d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:03.860913 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233d23c-1d32-475d-b0c2-5e5330f1395d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:03.861199 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233d23c-1d32-475d-b0c2-5e5330f1395d None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:03.861946 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8233d23c-1d32-475d-b0c2-5e5330f1395d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:03.862263 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8233d23c-1d32-475d-b0c2-5e5330f1395d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:03.863806 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8233d23c-1d32-475d-b0c2-5e5330f1395d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:03.876627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 538/1070] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:48:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:48:03.882535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8233d23c-1d32-475d-b0c2-5e5330f1395d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:03.894308 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:03.900291 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20de57-1ede-4514-867b-7918fa1f3a17 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:03.947554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e548f2-1469-4c36-9a04-a0ad71951a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:03.948751 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e548f2-1469-4c36-9a04-a0ad71951a14 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:03.949314 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e548f2-1469-4c36-9a04-a0ad71951a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:03.949598 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e548f2-1469-4c36-9a04-a0ad71951a14 None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:03.950590 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e548f2-1469-4c36-9a04-a0ad71951a14 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:03.950905 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e548f2-1469-4c36-9a04-a0ad71951a14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:03.952954 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e548f2-1469-4c36-9a04-a0ad71951a14 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:03.978045 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 533/1071] 10.222.0.46 () {56 vars in 1133 bytes} [Tue Aug 2 08:48:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:48:03.981569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e548f2-1469-4c36-9a04-a0ad71951a14 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:03.997595 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8233d23c-1d32-475d-b0c2-5e5330f1395d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:04.031312 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:04.036764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abae698-574e-474d-9293-6e439dc41769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:04.037412 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abae698-574e-474d-9293-6e439dc41769 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:04.037689 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abae698-574e-474d-9293-6e439dc41769 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:04.039261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abae698-574e-474d-9293-6e439dc41769 None admin] PATH_INFO: `/v3/users/6bfcb66257064f45ac3e08cbac49181c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:04.039261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abae698-574e-474d-9293-6e439dc41769 None admin] RBAC: Authorizing `identity:get_user(user_id=6bfcb66257064f45ac3e08cbac49181c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:04.040591 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abae698-574e-474d-9293-6e439dc41769 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:04.046929 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 539/1072] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:48:03 2022] GET /identity/v3/users/6bfcb66257064f45ac3e08cbac49181c => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:04.047700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abae698-574e-474d-9293-6e439dc41769 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:04.088452 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e548f2-1469-4c36-9a04-a0ad71951a14 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:04.124637 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cf15f-d657-4c78-ad7a-281d0213171a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:04.125271 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cf15f-d657-4c78-ad7a-281d0213171a None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:04.125549 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cf15f-d657-4c78-ad7a-281d0213171a None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:04.125818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cf15f-d657-4c78-ad7a-281d0213171a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:04.128971 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:04.136745 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cf15f-d657-4c78-ad7a-281d0213171a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:04.138065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cf15f-d657-4c78-ad7a-281d0213171a None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:04.152925 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abae698-574e-474d-9293-6e439dc41769 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:04.154065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-164cf15f-d657-4c78-ad7a-281d0213171a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:48:04.154065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:04.154065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:48:04.154065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:48:04.154065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:48:04.154065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:48:04.154065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:48:04.154065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:48:04.154065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:48:04.154065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:48:04.154065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:48:04.154065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:48:04.154065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:48:04.154065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:48:04.154065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:48:04.155002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:48:04.155002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:48:04.155002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:48:04.155002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:48:04.155002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:48:04.155002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:48:04.155002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:48:04.155002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:48:04.155002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:48:04.155002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:48:04.155002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:48:04.155002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:48:04.155002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:48:04.155002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:48:04.155002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:48:04.155002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:48:04.155002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:48:04.160583 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:48:04.160583 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:48:04.160583 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:48:04.160583 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:48:04.160583 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 534/1073] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:48:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:48:04.160938 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cf15f-d657-4c78-ad7a-281d0213171a None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:04.196870 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1c749-5845-4d9a-9ec8-e899d5d628c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:04.197513 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1c749-5845-4d9a-9ec8-e899d5d628c5 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:04.197794 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1c749-5845-4d9a-9ec8-e899d5d628c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:04.198095 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1c749-5845-4d9a-9ec8-e899d5d628c5 None admin] PATH_INFO: `/v3/projects/239afe35723747f6b978e5e6d5aede07` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:04.199503 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc1c749-5845-4d9a-9ec8-e899d5d628c5 None admin] RBAC: Authorizing `identity:get_project(project_id=239afe35723747f6b978e5e6d5aede07)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:04.201358 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc1c749-5845-4d9a-9ec8-e899d5d628c5 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:04.203611 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 540/1074] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:48:04 2022] GET /identity/v3/projects/239afe35723747f6b978e5e6d5aede07 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:04.209791 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1c749-5845-4d9a-9ec8-e899d5d628c5 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:04.259866 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:04.263717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cf15f-d657-4c78-ad7a-281d0213171a None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:04.298373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9d5ff-a252-4fc3-89aa-2e3311e228c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:04.298992 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9d5ff-a252-4fc3-89aa-2e3311e228c7 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:04.299273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9d5ff-a252-4fc3-89aa-2e3311e228c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:04.299548 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9d5ff-a252-4fc3-89aa-2e3311e228c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:04.300235 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9d5ff-a252-4fc3-89aa-2e3311e228c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:04.300547 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9d5ff-a252-4fc3-89aa-2e3311e228c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:04.301963 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9d5ff-a252-4fc3-89aa-2e3311e228c7 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:04.307340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1c749-5845-4d9a-9ec8-e899d5d628c5 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:04.313021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 535/1075] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:04.314802 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f9000-9c84-4b01-9b51-6c1d084e3e0e None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:04.315094 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f9000-9c84-4b01-9b51-6c1d084e3e0e None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:04.315377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f9000-9c84-4b01-9b51-6c1d084e3e0e None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:04.317587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 536/1076] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:48:04.318942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f9000-9c84-4b01-9b51-6c1d084e3e0e None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:04.346611 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8bf72-4988-414b-9e75-eab5253cb464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:04.347283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8bf72-4988-414b-9e75-eab5253cb464 None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:04.347562 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8bf72-4988-414b-9e75-eab5253cb464 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:04.347833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8bf72-4988-414b-9e75-eab5253cb464 None admin] PATH_INFO: `/v3/projects/239afe35723747f6b978e5e6d5aede07/users/6bfcb66257064f45ac3e08cbac49181c/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:04.350296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e8bf72-4988-414b-9e75-eab5253cb464 None admin] RBAC: Authorizing `identity:create_grant(project_id=239afe35723747f6b978e5e6d5aede07, user_id=6bfcb66257064f45ac3e08cbac49181c, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:04.351704 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e8bf72-4988-414b-9e75-eab5253cb464 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:04.419047 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 541/1077] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:48:04 2022] PUT /identity/v3/projects/239afe35723747f6b978e5e6d5aede07/users/6bfcb66257064f45ac3e08cbac49181c/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 08:48:04.428521 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a849f-4210-434d-b78f-717480afd4b6 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:04.428521 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a849f-4210-434d-b78f-717480afd4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:04.428521 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a849f-4210-434d-b78f-717480afd4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:04.428831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f9000-9c84-4b01-9b51-6c1d084e3e0e None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:04.430390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:04.442134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-7a4a849f-4210-434d-b78f-717480afd4b6 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:48:04.443386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7a4a849f-4210-434d-b78f-717480afd4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:48:04.469519 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed7a88-99e3-43f2-b378-5df9f7814069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:04.470456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed7a88-99e3-43f2-b378-5df9f7814069 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:04.470748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed7a88-99e3-43f2-b378-5df9f7814069 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:04.471035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed7a88-99e3-43f2-b378-5df9f7814069 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:04.481022 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ed7a88-99e3-43f2-b378-5df9f7814069 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:04.482274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ed7a88-99e3-43f2-b378-5df9f7814069 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:04.495660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-38ed7a88-99e3-43f2-b378-5df9f7814069 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:48:04.495660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:04.495660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:48:04.495660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:48:04.495660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:48:04.495660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:48:04.495660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:48:04.495660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:48:04.495660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:48:04.495660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:48:04.495660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:48:04.495660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:48:04.495660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:48:04.495660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:48:04.495660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:48:04.496680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:48:04.496680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:48:04.496680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:48:04.496680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:48:04.496680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:48:04.496680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:48:04.496680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:48:04.496680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:48:04.496680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:48:04.496680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:48:04.496680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:48:04.496680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:48:04.496680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:48:04.496680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:48:04.496680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:48:04.496680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:48:04.496680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:48:04.497619 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:48:04.497619 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:48:04.497619 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:48:04.497619 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:48:04.497619 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:48:04.497619 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:48:04.497619 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:48:04.497619 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:48:04.497619 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 537/1078] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:48:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:48:04.503258 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed7a88-99e3-43f2-b378-5df9f7814069 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:04.526755 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 542/1079] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:48:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:48:04.528785 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:04.549208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a849f-4210-434d-b78f-717480afd4b6 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:04.619154 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed7a88-99e3-43f2-b378-5df9f7814069 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:04.653679 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0da3a-6a98-401a-94a7-2991a58959d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:04.654574 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0da3a-6a98-401a-94a7-2991a58959d7 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:04.654855 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0da3a-6a98-401a-94a7-2991a58959d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:04.655135 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0da3a-6a98-401a-94a7-2991a58959d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:04.655885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a0da3a-6a98-401a-94a7-2991a58959d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:04.656192 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a0da3a-6a98-401a-94a7-2991a58959d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:04.657611 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a0da3a-6a98-401a-94a7-2991a58959d7 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:04.667475 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a849f-4210-434d-b78f-717480afd4b6 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:04.671806 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 538/1080] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:48:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:48:04.678570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0da3a-6a98-401a-94a7-2991a58959d7 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:04.706434 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e9b97-52d0-4040-8a15-0fa2290181ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:04.707061 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e9b97-52d0-4040-8a15-0fa2290181ba None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:04.707368 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e9b97-52d0-4040-8a15-0fa2290181ba None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:04.707612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e9b97-52d0-4040-8a15-0fa2290181ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:04.712253 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:04.761514 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:04.780563 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e9b97-52d0-4040-8a15-0fa2290181ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:04.782333 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e9b97-52d0-4040-8a15-0fa2290181ba None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:04.782787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0da3a-6a98-401a-94a7-2991a58959d7 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:04.818880 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186d66d-2bf5-4c7f-a40e-8d537c7e0211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:04.819589 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186d66d-2bf5-4c7f-a40e-8d537c7e0211 None admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:04.819872 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186d66d-2bf5-4c7f-a40e-8d537c7e0211 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:04.820158 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186d66d-2bf5-4c7f-a40e-8d537c7e0211 None admin] PATH_INFO: `/v3/projects/4227a5b804c0414c99d407bb1415cf49/users/3191d804e8dc4b1a85f490319833ec07/roles/1606ffdafce14431a59c0f6e8c862c1d` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:04.822424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186d66d-2bf5-4c7f-a40e-8d537c7e0211 None admin] RBAC: Authorizing `identity:create_grant(project_id=4227a5b804c0414c99d407bb1415cf49, user_id=3191d804e8dc4b1a85f490319833ec07, role_id=1606ffdafce14431a59c0f6e8c862c1d)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:04.824137 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186d66d-2bf5-4c7f-a40e-8d537c7e0211 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:04.872533 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 539/1081] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:48:04 2022] PUT /identity/v3/projects/4227a5b804c0414c99d407bb1415cf49/users/3191d804e8dc4b1a85f490319833ec07/roles/1606ffdafce14431a59c0f6e8c862c1d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 08:48:04.916556 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 543/1082] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:48:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 08:48:04.924014 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186d66d-2bf5-4c7f-a40e-8d537c7e0211 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:05.035059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186d66d-2bf5-4c7f-a40e-8d537c7e0211 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:05.071433 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab93ec5-befe-4b8e-b213-fd490eb8a7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:05.072322 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab93ec5-befe-4b8e-b213-fd490eb8a7cc None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:05.072775 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab93ec5-befe-4b8e-b213-fd490eb8a7cc None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:05.073210 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab93ec5-befe-4b8e-b213-fd490eb8a7cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:05.081276 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:05.088762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab93ec5-befe-4b8e-b213-fd490eb8a7cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:05.091402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab93ec5-befe-4b8e-b213-fd490eb8a7cc None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:05.106585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-3ab93ec5-befe-4b8e-b213-fd490eb8a7cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:48:05.106585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:05.106585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:48:05.106585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:48:05.106585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:48:05.106585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:48:05.106585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:48:05.106585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:48:05.106585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:48:05.106585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:48:05.106585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:48:05.106585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:48:05.106585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:48:05.106585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:48:05.106585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:48:05.107556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:48:05.107556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:48:05.107556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:48:05.107556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:48:05.107556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:48:05.107556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:48:05.107556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:48:05.107556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:48:05.107556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:48:05.107556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:48:05.107556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:48:05.107556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:48:05.107556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:48:05.107556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:48:05.107556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:48:05.107556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:48:05.107556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:48:05.113545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:48:05.113545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:48:05.113545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:48:05.113545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:48:05.113545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 540/1083] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:48:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:48:05.116101 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e9b97-52d0-4040-8a15-0fa2290181ba None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:05.151169 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:05.218174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e9b97-52d0-4040-8a15-0fa2290181ba None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:05.252786 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac56860a-4ca4-4475-aec1-4c66e76963ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:05.253404 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56860a-4ca4-4475-aec1-4c66e76963ed None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:05.253688 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56860a-4ca4-4475-aec1-4c66e76963ed None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:05.253991 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56860a-4ca4-4475-aec1-4c66e76963ed None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:05.255272 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac56860a-4ca4-4475-aec1-4c66e76963ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:05.255507 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac56860a-4ca4-4475-aec1-4c66e76963ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:05.256868 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac56860a-4ca4-4475-aec1-4c66e76963ed None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:05.268084 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 544/1084] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:06.435782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6a769-c60a-4fe9-8c96-88e660807f3e None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:06.436310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6a769-c60a-4fe9-8c96-88e660807f3e None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:06.436310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6a769-c60a-4fe9-8c96-88e660807f3e None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:06.437671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 541/1085] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:48:06.442943 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda8f9b-e319-4f49-9036-5fcb308ce2ed None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:06.443398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda8f9b-e319-4f49-9036-5fcb308ce2ed None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:06.443488 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda8f9b-e319-4f49-9036-5fcb308ce2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:06.461944 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5fda8f9b-e319-4f49-9036-5fcb308ce2ed None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:48:06.463134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5fda8f9b-e319-4f49-9036-5fcb308ce2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:48:06.529873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 545/1086] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:48:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:48:06.556726 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6a769-c60a-4fe9-8c96-88e660807f3e None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:06.657572 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6a769-c60a-4fe9-8c96-88e660807f3e None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:06.691075 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b61f89-41cf-4a5c-8c3d-0ccaaae5c5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:06.691641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b61f89-41cf-4a5c-8c3d-0ccaaae5c5cc None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:06.691926 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b61f89-41cf-4a5c-8c3d-0ccaaae5c5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:06.692715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b61f89-41cf-4a5c-8c3d-0ccaaae5c5cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:06.702853 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b61f89-41cf-4a5c-8c3d-0ccaaae5c5cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:48:06.708213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-02b61f89-41cf-4a5c-8c3d-0ccaaae5c5cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:48:06.708213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:06.708213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:48:06.708213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:48:06.708213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:48:06.708213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:48:06.708213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:48:06.708213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:48:06.708213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:48:06.708213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:48:06.708213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:48:06.708213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:48:06.708213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:48:06.708213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:48:06.708213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:48:06.708213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:48:06.709324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:48:06.709324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:48:06.709324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:48:06.709324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:48:06.709324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:48:06.709324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:48:06.709324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:48:06.709324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:48:06.709324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:48:06.709324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:48:06.709324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:48:06.709324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:48:06.709324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:48:06.709324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:48:06.709324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:48:06.709324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:48:06.715600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:48:06.715600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:48:06.715600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:48:06.715600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:48:06.715600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:48:06.715600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:48:06.715600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:48:06.715600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:48:06.715600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:48:06.715600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:48:06.715600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:48:06.715600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:48:06.715600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 542/1087] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:48:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:48:06.716627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda8f9b-e319-4f49-9036-5fcb308ce2ed None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:06.741783 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:06.814773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda8f9b-e319-4f49-9036-5fcb308ce2ed None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:06.848739 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cff1d8-9fbd-4559-b722-0bd8eb1e26fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:06.849358 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cff1d8-9fbd-4559-b722-0bd8eb1e26fd None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:06.849641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cff1d8-9fbd-4559-b722-0bd8eb1e26fd None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:06.849943 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cff1d8-9fbd-4559-b722-0bd8eb1e26fd None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:06.850659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cff1d8-9fbd-4559-b722-0bd8eb1e26fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:06.850965 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cff1d8-9fbd-4559-b722-0bd8eb1e26fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:06.852337 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cff1d8-9fbd-4559-b722-0bd8eb1e26fd None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:06.860443 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 546/1088] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:48:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:06.866832 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b61f89-41cf-4a5c-8c3d-0ccaaae5c5cc None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:06.888325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd56f0-14cd-4599-9eb8-2d27177360ef None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:06.888737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd56f0-14cd-4599-9eb8-2d27177360ef None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:06.889021 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd56f0-14cd-4599-9eb8-2d27177360ef None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:06.890391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 547/1089] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:48:06.895914 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c98282-c758-4c46-8038-5df36d4137f3 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:06.896202 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c98282-c758-4c46-8038-5df36d4137f3 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:06.896483 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c98282-c758-4c46-8038-5df36d4137f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:06.914779 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-d3c98282-c758-4c46-8038-5df36d4137f3 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:48:06.915983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d3c98282-c758-4c46-8038-5df36d4137f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:48:06.917303 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:06.970585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b61f89-41cf-4a5c-8c3d-0ccaaae5c5cc None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:06.986968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 548/1090] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:48:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 08:48:07.003760 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd968a-5a89-4171-9e8e-fd7b2184d5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:07.004393 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd968a-5a89-4171-9e8e-fd7b2184d5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:07.005251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd968a-5a89-4171-9e8e-fd7b2184d5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:07.005525 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd968a-5a89-4171-9e8e-fd7b2184d5a9 None admin] PATH_INFO: `/v3/users/6bfcb66257064f45ac3e08cbac49181c` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:07.006695 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd968a-5a89-4171-9e8e-fd7b2184d5a9 None admin] RBAC: Authorizing `identity:get_user(user_id=6bfcb66257064f45ac3e08cbac49181c)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:07.008105 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd968a-5a89-4171-9e8e-fd7b2184d5a9 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:07.009989 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 543/1091] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:48:06 2022] GET /identity/v3/users/6bfcb66257064f45ac3e08cbac49181c => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:07.010928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c98282-c758-4c46-8038-5df36d4137f3 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:07.015198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd968a-5a89-4171-9e8e-fd7b2184d5a9 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:07.028233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:07.108068 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c98282-c758-4c46-8038-5df36d4137f3 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:07.119076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd968a-5a89-4171-9e8e-fd7b2184d5a9 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:07.135930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:07.144953 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c58548-2265-46a8-a74b-32bb5ca2189e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:07.149707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c58548-2265-46a8-a74b-32bb5ca2189e None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:07.149707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c58548-2265-46a8-a74b-32bb5ca2189e None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:07.149707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c58548-2265-46a8-a74b-32bb5ca2189e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:07.154303 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91dfb8-b302-4a54-aa3f-efb5ee9e5bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:07.154919 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91dfb8-b302-4a54-aa3f-efb5ee9e5bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:07.155201 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91dfb8-b302-4a54-aa3f-efb5ee9e5bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:07.155478 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91dfb8-b302-4a54-aa3f-efb5ee9e5bd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:07.158850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c58548-2265-46a8-a74b-32bb5ca2189e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:48:07.164444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-85c58548-2265-46a8-a74b-32bb5ca2189e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 08:48:07.164444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:07.164444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:48:07.164444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:48:07.164444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:48:07.164444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:48:07.164444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:48:07.164444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:48:07.164444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:48:07.164444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:48:07.164444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:48:07.164444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:48:07.164444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:48:07.164444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:48:07.164444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:48:07.164444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:48:07.165381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:48:07.165381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:48:07.165381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:48:07.165381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:48:07.165381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:48:07.165381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:48:07.165381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:48:07.165381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:48:07.165381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:48:07.165381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:48:07.165381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:48:07.165381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:48:07.165381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:48:07.165381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:48:07.165381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:48:07.165381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:48:07.166337 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:48:07.166337 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:48:07.166337 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:48:07.166337 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:48:07.166337 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:48:07.166337 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:48:07.166337 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:48:07.166337 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:48:07.166337 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:48:07.166337 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:48:07.166337 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 08:48:07.166337 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:48:07.166337 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 549/1092] 10.222.0.46 () {56 vars in 1120 bytes} [Tue Aug 2 08:48:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:48:07.171686 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c58548-2265-46a8-a74b-32bb5ca2189e None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:07.224691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee91dfb8-b302-4a54-aa3f-efb5ee9e5bd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:07.226082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee91dfb8-b302-4a54-aa3f-efb5ee9e5bd2 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:07.248831 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:07.271122 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c58548-2265-46a8-a74b-32bb5ca2189e None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:07.305349 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf108d30-8299-4d46-b89c-f65db63a17a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:07.305996 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf108d30-8299-4d46-b89c-f65db63a17a3 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:07.306281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf108d30-8299-4d46-b89c-f65db63a17a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:07.306560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf108d30-8299-4d46-b89c-f65db63a17a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:07.307309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf108d30-8299-4d46-b89c-f65db63a17a3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:07.307617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf108d30-8299-4d46-b89c-f65db63a17a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:07.308986 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf108d30-8299-4d46-b89c-f65db63a17a3 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:07.316886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 550/1093] 10.222.0.46 () {56 vars in 1115 bytes} [Tue Aug 2 08:48:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:48:07.322422 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf108d30-8299-4d46-b89c-f65db63a17a3 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:07.329553 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 544/1094] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:48:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 08:48:07.330594 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91dfb8-b302-4a54-aa3f-efb5ee9e5bd2 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:07.347821 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:07.425679 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf108d30-8299-4d46-b89c-f65db63a17a3 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:07.425981 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91dfb8-b302-4a54-aa3f-efb5ee9e5bd2 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:07.460681 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fce04d-0101-4887-9478-6846276e9b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:07.461352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b067f35-2f35-4489-a4b9-023d2417be35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:07.461724 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fce04d-0101-4887-9478-6846276e9b7a None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:07.461818 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b067f35-2f35-4489-a4b9-023d2417be35 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:07.461937 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fce04d-0101-4887-9478-6846276e9b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:07.462044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b067f35-2f35-4489-a4b9-023d2417be35 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:07.462132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fce04d-0101-4887-9478-6846276e9b7a None admin] PATH_INFO: `/v3/projects/239afe35723747f6b978e5e6d5aede07` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:07.462217 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b067f35-2f35-4489-a4b9-023d2417be35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:07.463036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fce04d-0101-4887-9478-6846276e9b7a None admin] RBAC: Authorizing `identity:get_project(project_id=239afe35723747f6b978e5e6d5aede07)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:07.464474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fce04d-0101-4887-9478-6846276e9b7a None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:07.466186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 545/1095] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:48:07 2022] GET /identity/v3/projects/239afe35723747f6b978e5e6d5aede07 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:48:07.470645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:07.472077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fce04d-0101-4887-9478-6846276e9b7a None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:07.475312 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b067f35-2f35-4489-a4b9-023d2417be35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:07.476674 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b067f35-2f35-4489-a4b9-023d2417be35 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:07.491370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-8b067f35-2f35-4489-a4b9-023d2417be35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:48:07.491370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:07.491370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:48:07.491370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:48:07.491370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:48:07.491370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:48:07.491370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:48:07.491370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:48:07.491370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:48:07.491370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:48:07.491370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:48:07.491370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:48:07.491370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:48:07.491370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:48:07.491370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:48:07.492289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:48:07.492289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:48:07.492289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:48:07.492289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:48:07.492289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:48:07.492289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:48:07.492289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:48:07.492289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:48:07.492289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:48:07.492289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:48:07.492289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:48:07.492289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:48:07.492289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:48:07.492289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:48:07.492289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:48:07.492289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:48:07.492289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:48:07.493225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:48:07.493225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:48:07.493225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:48:07.493225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:48:07.493225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 551/1096] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:48:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:48:07.499103 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b067f35-2f35-4489-a4b9-023d2417be35 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:07.560364 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:07.574456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fce04d-0101-4887-9478-6846276e9b7a None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:07.600867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b067f35-2f35-4489-a4b9-023d2417be35 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:07.608438 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381c80a-ab7e-44cd-b37f-97a55124706a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:07.609059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381c80a-ab7e-44cd-b37f-97a55124706a None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:07.609348 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381c80a-ab7e-44cd-b37f-97a55124706a None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:07.609619 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381c80a-ab7e-44cd-b37f-97a55124706a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:07.610436 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9381c80a-ab7e-44cd-b37f-97a55124706a None admin] RBAC: Adding query filter params (role.id=8ce0850e15804f3d811e88cc9841e673, scope.project.id=239afe35723747f6b978e5e6d5aede07, user.id=6bfcb66257064f45ac3e08cbac49181c) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:07.610746 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9381c80a-ab7e-44cd-b37f-97a55124706a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:07.612160 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9381c80a-ab7e-44cd-b37f-97a55124706a None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:07.618527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 546/1097] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:48:07 2022] GET /identity/v3/role_assignments?user.id=6bfcb66257064f45ac3e08cbac49181c&scope.project.id=239afe35723747f6b978e5e6d5aede07&role.id=8ce0850e15804f3d811e88cc9841e673 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:07.635549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7e60d-9342-4cae-a161-8dc47a458b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:07.636196 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7e60d-9342-4cae-a161-8dc47a458b16 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:07.636477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7e60d-9342-4cae-a161-8dc47a458b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:07.636749 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b7e60d-9342-4cae-a161-8dc47a458b16 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:07.637458 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b7e60d-9342-4cae-a161-8dc47a458b16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:07.637766 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b7e60d-9342-4cae-a161-8dc47a458b16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:07.639214 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b7e60d-9342-4cae-a161-8dc47a458b16 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:07.652902 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 552/1098] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:48:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:48:07.658680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381c80a-ab7e-44cd-b37f-97a55124706a None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:07.762427 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9381c80a-ab7e-44cd-b37f-97a55124706a None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:07.796697 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c460807-cdb9-47fb-bdab-b672bf1d9310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:07.797331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c460807-cdb9-47fb-bdab-b672bf1d9310 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:07.797605 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c460807-cdb9-47fb-bdab-b672bf1d9310 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:07.797874 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c460807-cdb9-47fb-bdab-b672bf1d9310 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:07.805869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c460807-cdb9-47fb-bdab-b672bf1d9310 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:07.807317 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c460807-cdb9-47fb-bdab-b672bf1d9310 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:07.818987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-5c460807-cdb9-47fb-bdab-b672bf1d9310 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:48:07.818987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:07.818987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:48:07.818987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:48:07.818987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:48:07.818987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:48:07.818987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:48:07.818987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:48:07.818987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:48:07.818987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:48:07.818987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:48:07.818987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:48:07.818987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:48:07.818987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:48:07.818987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:48:07.818987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:48:07.823916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:48:07.823916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:48:07.823916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:48:07.823916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:48:07.823916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:48:07.823916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:48:07.823916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:48:07.823916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:48:07.823916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:48:07.823916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:48:07.823916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:48:07.823916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:48:07.823916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:48:07.823916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:48:07.823916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:48:07.823916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:48:07.823916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:48:07.823916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:48:07.825387 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:48:07.825387 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:48:07.825387 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:48:07.825387 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:48:07.825387 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:48:07.825387 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:48:07.825387 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:48:07.825387 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:48:07.825387 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 08:48:07.825387 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:48:07.825387 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 547/1099] 10.222.0.46 () {56 vars in 1093 bytes} [Tue Aug 2 08:48:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:48:07.827267 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7e60d-9342-4cae-a161-8dc47a458b16 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:07.880089 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:07.931065 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b7e60d-9342-4cae-a161-8dc47a458b16 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:07.965559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e29cf3-473a-4b93-a3bb-ca786ef7a24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:07.966201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e29cf3-473a-4b93-a3bb-ca786ef7a24b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:07.966478 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e29cf3-473a-4b93-a3bb-ca786ef7a24b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:07.966748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e29cf3-473a-4b93-a3bb-ca786ef7a24b None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:07.967531 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e29cf3-473a-4b93-a3bb-ca786ef7a24b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:07.967836 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e29cf3-473a-4b93-a3bb-ca786ef7a24b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:07.969287 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e29cf3-473a-4b93-a3bb-ca786ef7a24b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:07.986564 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 553/1100] 10.222.0.46 () {56 vars in 1133 bytes} [Tue Aug 2 08:48:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:07.993116 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c460807-cdb9-47fb-bdab-b672bf1d9310 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:08.005725 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:08.090926 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c460807-cdb9-47fb-bdab-b672bf1d9310 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:08.124084 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c191746-77f3-47d9-94b3-4814ff8d88e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:08.124690 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c191746-77f3-47d9-94b3-4814ff8d88e5 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:08.124968 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c191746-77f3-47d9-94b3-4814ff8d88e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:08.125238 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c191746-77f3-47d9-94b3-4814ff8d88e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:08.191254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c191746-77f3-47d9-94b3-4814ff8d88e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:08.192704 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c191746-77f3-47d9-94b3-4814ff8d88e5 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:08.203733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:08.307539 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 548/1101] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:48:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 08:48:08.313280 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e29cf3-473a-4b93-a3bb-ca786ef7a24b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:08.410788 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e29cf3-473a-4b93-a3bb-ca786ef7a24b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:08.419667 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:08.444008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41980be8-d7bc-4b7f-bb42-b1f1a00d0ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:08.444640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41980be8-d7bc-4b7f-bb42-b1f1a00d0ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:08.444917 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41980be8-d7bc-4b7f-bb42-b1f1a00d0ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:08.445187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41980be8-d7bc-4b7f-bb42-b1f1a00d0ce8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:08.455476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41980be8-d7bc-4b7f-bb42-b1f1a00d0ce8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:08.456906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41980be8-d7bc-4b7f-bb42-b1f1a00d0ce8 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:08.470902 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-41980be8-d7bc-4b7f-bb42-b1f1a00d0ce8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:48:08.470902 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:08.470902 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:48:08.470902 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:48:08.470902 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:48:08.470902 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:48:08.470902 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:48:08.470902 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:48:08.470902 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:48:08.470902 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:48:08.470902 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:48:08.470902 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:48:08.470902 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:48:08.470902 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:48:08.470902 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:48:08.477329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:48:08.477329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:48:08.477329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:48:08.477329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:48:08.477329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:48:08.477329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:48:08.477329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:48:08.477329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:48:08.477329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:48:08.477329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:48:08.477329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:48:08.477329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:48:08.477329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:48:08.477329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:48:08.477329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:48:08.477329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:48:08.477329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:48:08.478495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:48:08.478495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:48:08.478495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:48:08.478495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:48:08.478495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 554/1102] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:48:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:48:08.478784 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c191746-77f3-47d9-94b3-4814ff8d88e5 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:08.576434 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c191746-77f3-47d9-94b3-4814ff8d88e5 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:08.587286 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:08.610747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44175bcb-a43d-48c2-9be0-e91ce9423fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:08.611364 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44175bcb-a43d-48c2-9be0-e91ce9423fce None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:08.611647 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44175bcb-a43d-48c2-9be0-e91ce9423fce None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:08.611922 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44175bcb-a43d-48c2-9be0-e91ce9423fce None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:08.612608 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44175bcb-a43d-48c2-9be0-e91ce9423fce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:08.612918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44175bcb-a43d-48c2-9be0-e91ce9423fce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:08.615221 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44175bcb-a43d-48c2-9be0-e91ce9423fce None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:08.627469 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 549/1103] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:48:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:08.633822 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41980be8-d7bc-4b7f-bb42-b1f1a00d0ce8 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:08.730311 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41980be8-d7bc-4b7f-bb42-b1f1a00d0ce8 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:08.763472 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c7a58-1a47-481f-a60f-e001b3d61d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:08.764133 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c7a58-1a47-481f-a60f-e001b3d61d9c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:08.764413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c7a58-1a47-481f-a60f-e001b3d61d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:08.764689 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c7a58-1a47-481f-a60f-e001b3d61d9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:08.772138 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c7a58-1a47-481f-a60f-e001b3d61d9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:08.773586 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c7a58-1a47-481f-a60f-e001b3d61d9c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:08.784928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-2d2c7a58-1a47-481f-a60f-e001b3d61d9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:48:08.784928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:08.784928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:48:08.784928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:48:08.784928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:48:08.784928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:48:08.784928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:48:08.784928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:48:08.784928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:48:08.784928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:48:08.784928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:48:08.784928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:48:08.784928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:48:08.784928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:48:08.784928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:48:08.786317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:48:08.786317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:48:08.786317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:48:08.786317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:48:08.786317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:48:08.786317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:48:08.786317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:48:08.786317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:48:08.786317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:48:08.786317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:48:08.786317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:48:08.786317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:48:08.786317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:48:08.786317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:48:08.786317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:48:08.786317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:48:08.786317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:48:08.787244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:48:08.787244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:48:08.787244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:48:08.787244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:48:08.787244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:48:08.787244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:48:08.787244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:48:08.787244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:48:08.787244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 555/1104] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:48:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:48:08.792372 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44175bcb-a43d-48c2-9be0-e91ce9423fce None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:08.927604 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44175bcb-a43d-48c2-9be0-e91ce9423fce None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:08.964740 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a2e38-9ba4-450e-a0d1-64e511b3af5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:08.965358 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a2e38-9ba4-450e-a0d1-64e511b3af5d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:08.965641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a2e38-9ba4-450e-a0d1-64e511b3af5d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:08.965941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a2e38-9ba4-450e-a0d1-64e511b3af5d None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:08.966691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a2e38-9ba4-450e-a0d1-64e511b3af5d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:08.967000 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a2e38-9ba4-450e-a0d1-64e511b3af5d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:08.968447 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a2e38-9ba4-450e-a0d1-64e511b3af5d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:08.980123 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 550/1105] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:48:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:08.986229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c7a58-1a47-481f-a60f-e001b3d61d9c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:09.008335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:09.083929 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c7a58-1a47-481f-a60f-e001b3d61d9c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:09.116402 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c660e-2281-4bf0-a3c0-ec1c82eda86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:09.117011 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c660e-2281-4bf0-a3c0-ec1c82eda86b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:09.117292 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c660e-2281-4bf0-a3c0-ec1c82eda86b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:09.117566 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c660e-2281-4bf0-a3c0-ec1c82eda86b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:09.118368 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c660e-2281-4bf0-a3c0-ec1c82eda86b None admin] RBAC: Adding query filter params (role.id=1606ffdafce14431a59c0f6e8c862c1d, scope.project.id=4227a5b804c0414c99d407bb1415cf49, user.id=3191d804e8dc4b1a85f490319833ec07) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:09.118679 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c660e-2281-4bf0-a3c0-ec1c82eda86b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:09.120098 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c660e-2281-4bf0-a3c0-ec1c82eda86b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:09.137927 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 556/1106] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:48:08 2022] GET /identity/v3/role_assignments?user.id=3191d804e8dc4b1a85f490319833ec07&scope.project.id=4227a5b804c0414c99d407bb1415cf49&role.id=1606ffdafce14431a59c0f6e8c862c1d => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:09.681324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321c9e1-8152-49e9-abd1-fb736bf8e967 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:09.681867 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321c9e1-8152-49e9-abd1-fb736bf8e967 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:09.681867 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7321c9e1-8152-49e9-abd1-fb736bf8e967 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:09.683274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 551/1107] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:48:09.688599 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669af9f-91ee-4ffa-84a5-91f5424c5e62 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:09.689006 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669af9f-91ee-4ffa-84a5-91f5424c5e62 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:09.689162 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669af9f-91ee-4ffa-84a5-91f5424c5e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:09.708463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-2669af9f-91ee-4ffa-84a5-91f5424c5e62 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:48:09.709669 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2669af9f-91ee-4ffa-84a5-91f5424c5e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:48:09.778870 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 557/1108] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:48:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:48:09.803260 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321c9e1-8152-49e9-abd1-fb736bf8e967 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:09.835223 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:09.908995 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7321c9e1-8152-49e9-abd1-fb736bf8e967 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:09.946359 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feabe7f3-404a-4e68-875a-edd538d440be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:09.947011 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabe7f3-404a-4e68-875a-edd538d440be None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:09.947292 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabe7f3-404a-4e68-875a-edd538d440be None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:09.947563 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feabe7f3-404a-4e68-875a-edd538d440be None admin] PATH_INFO: `/v3/domains/default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:09.948973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feabe7f3-404a-4e68-875a-edd538d440be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:09.950437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feabe7f3-404a-4e68-875a-edd538d440be None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:09.952044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 552/1109] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:48:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:09.958219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2669af9f-91ee-4ffa-84a5-91f5424c5e62 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:09.973057 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:10.053538 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2669af9f-91ee-4ffa-84a5-91f5424c5e62 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:10.086229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c0afe-13fd-4f62-b09d-04e83d7d1a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:10.086830 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c0afe-13fd-4f62-b09d-04e83d7d1a71 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:10.087108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c0afe-13fd-4f62-b09d-04e83d7d1a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:10.087380 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c0afe-13fd-4f62-b09d-04e83d7d1a71 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:10.088056 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c0afe-13fd-4f62-b09d-04e83d7d1a71 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:10.089469 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c0afe-13fd-4f62-b09d-04e83d7d1a71 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:10.201647 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 558/1110] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 2 08:48:09 2022] POST /identity/v3/users => generated 286 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:48:12.231062 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3a03e-7247-4059-8569-8e2c5002c5d3 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:12.231566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3a03e-7247-4059-8569-8e2c5002c5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:12.231566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd3a03e-7247-4059-8569-8e2c5002c5d3 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:12.232881 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 553/1111] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:48:12.238101 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88618b6-002b-40fe-bae2-5cd0c74cf05e None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:12.238496 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88618b6-002b-40fe-bae2-5cd0c74cf05e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:12.238652 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88618b6-002b-40fe-bae2-5cd0c74cf05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:12.257951 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-d88618b6-002b-40fe-bae2-5cd0c74cf05e None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:48:12.259123 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d88618b6-002b-40fe-bae2-5cd0c74cf05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:48:12.319744 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:12.324429 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 559/1112] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:48:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:48:12.347083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3a03e-7247-4059-8569-8e2c5002c5d3 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:12.441047 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:12.446017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd3a03e-7247-4059-8569-8e2c5002c5d3 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:12.479307 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b320306-002f-4d72-9ef4-9ae0ec4ca55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:12.480022 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b320306-002f-4d72-9ef4-9ae0ec4ca55a None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:12.480307 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b320306-002f-4d72-9ef4-9ae0ec4ca55a None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:12.480579 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b320306-002f-4d72-9ef4-9ae0ec4ca55a None admin] PATH_INFO: `/v3/roles/111e800ddb0a411c80664cd51ce13fc6` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:12.482081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b320306-002f-4d72-9ef4-9ae0ec4ca55a None admin] RBAC: Authorizing `identity:get_role(role_id=111e800ddb0a411c80664cd51ce13fc6)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:12.483387 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b320306-002f-4d72-9ef4-9ae0ec4ca55a None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:12.484710 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 554/1113] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:48:12 2022] GET /identity/v3/roles/111e800ddb0a411c80664cd51ce13fc6 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:12.491154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88618b6-002b-40fe-bae2-5cd0c74cf05e None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:12.586521 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88618b6-002b-40fe-bae2-5cd0c74cf05e None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:12.618482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e490b-c43c-47eb-b669-7cc003b9df37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:12.619115 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e490b-c43c-47eb-b669-7cc003b9df37 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:12.619397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e490b-c43c-47eb-b669-7cc003b9df37 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:12.619673 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e490b-c43c-47eb-b669-7cc003b9df37 None admin] PATH_INFO: `/v3/users/b78ed511cb474ce2b31c9b226c5a29ac` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:12.632593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e490b-c43c-47eb-b669-7cc003b9df37 None admin] RBAC: Authorizing `identity:get_user(user_id=b78ed511cb474ce2b31c9b226c5a29ac)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:12.634055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e490b-c43c-47eb-b669-7cc003b9df37 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:12.635658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 560/1114] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:48:12 2022] GET /identity/v3/users/b78ed511cb474ce2b31c9b226c5a29ac => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:12.641353 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b320306-002f-4d72-9ef4-9ae0ec4ca55a None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:12.737227 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b320306-002f-4d72-9ef4-9ae0ec4ca55a None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:12.769754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7268d-4154-40d0-ae09-1380e992fac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:12.770943 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7268d-4154-40d0-ae09-1380e992fac6 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:12.771224 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7268d-4154-40d0-ae09-1380e992fac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:12.771504 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7268d-4154-40d0-ae09-1380e992fac6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:12.803038 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:12.836713 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c7268d-4154-40d0-ae09-1380e992fac6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:12.838590 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c7268d-4154-40d0-ae09-1380e992fac6 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:12.933259 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 555/1115] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:48:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:48:12.939147 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e490b-c43c-47eb-b669-7cc003b9df37 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:12.997329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:13.038298 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e490b-c43c-47eb-b669-7cc003b9df37 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:13.071229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98856be3-4c30-4e0f-860a-5a918858ab5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:13.071862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98856be3-4c30-4e0f-860a-5a918858ab5b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:13.072146 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98856be3-4c30-4e0f-860a-5a918858ab5b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:13.072422 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98856be3-4c30-4e0f-860a-5a918858ab5b None admin] PATH_INFO: `/v3/projects/239afe35723747f6b978e5e6d5aede07` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:13.073825 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98856be3-4c30-4e0f-860a-5a918858ab5b None admin] RBAC: Authorizing `identity:get_project(project_id=239afe35723747f6b978e5e6d5aede07)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:13.075283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98856be3-4c30-4e0f-860a-5a918858ab5b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:13.076923 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 561/1116] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:48:12 2022] GET /identity/v3/projects/239afe35723747f6b978e5e6d5aede07 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:13.083675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7268d-4154-40d0-ae09-1380e992fac6 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:13.148096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:13.184962 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7268d-4154-40d0-ae09-1380e992fac6 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:13.233108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b897742-3a4c-4b8a-86a5-b39e710d18a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:13.233701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b897742-3a4c-4b8a-86a5-b39e710d18a7 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:13.234031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b897742-3a4c-4b8a-86a5-b39e710d18a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:13.234310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b897742-3a4c-4b8a-86a5-b39e710d18a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:13.235087 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b897742-3a4c-4b8a-86a5-b39e710d18a7 None admin] RBAC: Adding query filter params (role.id=111e800ddb0a411c80664cd51ce13fc6, scope.project.id=239afe35723747f6b978e5e6d5aede07, user.id=b78ed511cb474ce2b31c9b226c5a29ac) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:13.235400 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b897742-3a4c-4b8a-86a5-b39e710d18a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:13.236901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b897742-3a4c-4b8a-86a5-b39e710d18a7 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:13.242373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 556/1117] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:48:13 2022] GET /identity/v3/role_assignments?user.id=b78ed511cb474ce2b31c9b226c5a29ac&scope.project.id=239afe35723747f6b978e5e6d5aede07&role.id=111e800ddb0a411c80664cd51ce13fc6 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:15.208461 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c456a-a5d4-49b7-bd3d-00e2d780dab3 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:15.208979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c456a-a5d4-49b7-bd3d-00e2d780dab3 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:15.208979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c456a-a5d4-49b7-bd3d-00e2d780dab3 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:15.210322 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 562/1118] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:48:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:48:15.215626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a5467-9d56-40bf-8a89-71b566db5c71 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:15.216054 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a5467-9d56-40bf-8a89-71b566db5c71 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:15.216164 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3a5467-9d56-40bf-8a89-71b566db5c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:15.234496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-0b3a5467-9d56-40bf-8a89-71b566db5c71 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:48:15.235692 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-0b3a5467-9d56-40bf-8a89-71b566db5c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:48:15.300747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 557/1119] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:48:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:48:15.324784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c456a-a5d4-49b7-bd3d-00e2d780dab3 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:15.425000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c456a-a5d4-49b7-bd3d-00e2d780dab3 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:15.449766 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:15.461227 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9fb8b-9a1f-4337-9b8f-ff2445b4a052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:15.461227 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9fb8b-9a1f-4337-9b8f-ff2445b4a052 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:15.461227 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9fb8b-9a1f-4337-9b8f-ff2445b4a052 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:15.461227 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9fb8b-9a1f-4337-9b8f-ff2445b4a052 None admin] PATH_INFO: `/v3/roles/111e800ddb0a411c80664cd51ce13fc6` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:15.461840 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9fb8b-9a1f-4337-9b8f-ff2445b4a052 None admin] RBAC: Authorizing `identity:get_role(role_id=111e800ddb0a411c80664cd51ce13fc6)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:15.463242 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb9fb8b-9a1f-4337-9b8f-ff2445b4a052 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:15.464560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 563/1120] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:48:15 2022] GET /identity/v3/roles/111e800ddb0a411c80664cd51ce13fc6 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:15.470591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a5467-9d56-40bf-8a89-71b566db5c71 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:15.529182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:15.568664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3a5467-9d56-40bf-8a89-71b566db5c71 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:15.601823 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd7f99-790d-4641-a98b-1506907678bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:15.602489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd7f99-790d-4641-a98b-1506907678bb None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:15.602775 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd7f99-790d-4641-a98b-1506907678bb None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:15.603052 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd7f99-790d-4641-a98b-1506907678bb None admin] PATH_INFO: `/v3/users/b78ed511cb474ce2b31c9b226c5a29ac` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:15.604215 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd7f99-790d-4641-a98b-1506907678bb None admin] RBAC: Authorizing `identity:get_user(user_id=b78ed511cb474ce2b31c9b226c5a29ac)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:15.605651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd7f99-790d-4641-a98b-1506907678bb None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:15.607311 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 558/1121] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:48:15 2022] GET /identity/v3/users/b78ed511cb474ce2b31c9b226c5a29ac => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:15.613140 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9fb8b-9a1f-4337-9b8f-ff2445b4a052 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:15.709894 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9fb8b-9a1f-4337-9b8f-ff2445b4a052 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:15.742241 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d049f-1e55-47d8-8ba9-db01df20622a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:15.742810 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d049f-1e55-47d8-8ba9-db01df20622a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:15.743092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d049f-1e55-47d8-8ba9-db01df20622a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:15.743369 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d049f-1e55-47d8-8ba9-db01df20622a None admin] PATH_INFO: `/v3/projects/239afe35723747f6b978e5e6d5aede07` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:15.744503 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d049f-1e55-47d8-8ba9-db01df20622a None admin] RBAC: Authorizing `identity:get_project(project_id=239afe35723747f6b978e5e6d5aede07)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:15.745947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056d049f-1e55-47d8-8ba9-db01df20622a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:15.747575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 564/1122] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:48:15 2022] GET /identity/v3/projects/239afe35723747f6b978e5e6d5aede07 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:15.753469 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd7f99-790d-4641-a98b-1506907678bb None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:15.848929 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd7f99-790d-4641-a98b-1506907678bb None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:15.874122 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:15.882356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664dae10-d3b6-4ad2-ba8d-7ad73127fee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:15.889495 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dae10-d3b6-4ad2-ba8d-7ad73127fee6 None admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:15.889495 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dae10-d3b6-4ad2-ba8d-7ad73127fee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:15.889495 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dae10-d3b6-4ad2-ba8d-7ad73127fee6 None admin] PATH_INFO: `/v3/projects/239afe35723747f6b978e5e6d5aede07/users/b78ed511cb474ce2b31c9b226c5a29ac/roles/111e800ddb0a411c80664cd51ce13fc6` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:15.889495 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664dae10-d3b6-4ad2-ba8d-7ad73127fee6 None admin] RBAC: Authorizing `identity:create_grant(project_id=239afe35723747f6b978e5e6d5aede07, user_id=b78ed511cb474ce2b31c9b226c5a29ac, role_id=111e800ddb0a411c80664cd51ce13fc6)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:15.889495 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664dae10-d3b6-4ad2-ba8d-7ad73127fee6 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:15.990132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 559/1123] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:48:15 2022] PUT /identity/v3/projects/239afe35723747f6b978e5e6d5aede07/users/b78ed511cb474ce2b31c9b226c5a29ac/roles/111e800ddb0a411c80664cd51ce13fc6 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:48:17.961247 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edaa0a5-a448-45b1-ad8f-133e8244b510 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:17.961719 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edaa0a5-a448-45b1-ad8f-133e8244b510 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:17.961830 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edaa0a5-a448-45b1-ad8f-133e8244b510 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:17.963186 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 565/1124] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:48:17.968240 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f4392-f086-4e7a-882f-fc547af96a3f None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:17.968675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f4392-f086-4e7a-882f-fc547af96a3f None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:17.968776 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f4392-f086-4e7a-882f-fc547af96a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:17.986245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-6e0f4392-f086-4e7a-882f-fc547af96a3f None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:48:17.987407 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-6e0f4392-f086-4e7a-882f-fc547af96a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:48:18.050264 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 560/1125] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:48:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:48:18.073867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edaa0a5-a448-45b1-ad8f-133e8244b510 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:18.135401 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:18.172828 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edaa0a5-a448-45b1-ad8f-133e8244b510 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:18.206952 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ea94d-e879-4082-b8dd-ab098a575aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:18.207611 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ea94d-e879-4082-b8dd-ab098a575aef None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:18.207893 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ea94d-e879-4082-b8dd-ab098a575aef None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:18.208172 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ea94d-e879-4082-b8dd-ab098a575aef None admin] PATH_INFO: `/v3/roles/111e800ddb0a411c80664cd51ce13fc6` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:18.209613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ea94d-e879-4082-b8dd-ab098a575aef None admin] RBAC: Authorizing `identity:get_role(role_id=111e800ddb0a411c80664cd51ce13fc6)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:18.211008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ea94d-e879-4082-b8dd-ab098a575aef None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:18.212325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 566/1126] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:48:18 2022] GET /identity/v3/roles/111e800ddb0a411c80664cd51ce13fc6 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:18.218143 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f4392-f086-4e7a-882f-fc547af96a3f None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:18.314585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f4392-f086-4e7a-882f-fc547af96a3f None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:18.332054 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:18.348164 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f2a68-23e7-4d92-9d07-3d8072af707b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:18.348790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f2a68-23e7-4d92-9d07-3d8072af707b None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:18.349077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f2a68-23e7-4d92-9d07-3d8072af707b None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:18.349351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f2a68-23e7-4d92-9d07-3d8072af707b None admin] PATH_INFO: `/v3/users/b78ed511cb474ce2b31c9b226c5a29ac` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:18.350503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f2a68-23e7-4d92-9d07-3d8072af707b None admin] RBAC: Authorizing `identity:get_user(user_id=b78ed511cb474ce2b31c9b226c5a29ac)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:18.351918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f2a68-23e7-4d92-9d07-3d8072af707b None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:18.353496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 561/1127] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:48:18 2022] GET /identity/v3/users/b78ed511cb474ce2b31c9b226c5a29ac => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:18.359178 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ea94d-e879-4082-b8dd-ab098a575aef None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:18.457504 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ea94d-e879-4082-b8dd-ab098a575aef None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:18.485003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:18.491105 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b2d0b-0eb0-46ea-b9f2-ebb78ca4634d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:18.491704 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b2d0b-0eb0-46ea-b9f2-ebb78ca4634d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:18.491981 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b2d0b-0eb0-46ea-b9f2-ebb78ca4634d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:18.492252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b2d0b-0eb0-46ea-b9f2-ebb78ca4634d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:18.557721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b2d0b-0eb0-46ea-b9f2-ebb78ca4634d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:18.559177 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b2d0b-0eb0-46ea-b9f2-ebb78ca4634d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:18.653653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 567/1128] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:48:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:48:18.659586 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f2a68-23e7-4d92-9d07-3d8072af707b None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:18.757293 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f2a68-23e7-4d92-9d07-3d8072af707b None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:18.789164 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d61052e-6846-4569-aaab-1b5810f62f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:18.789788 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61052e-6846-4569-aaab-1b5810f62f99 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:18.790128 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61052e-6846-4569-aaab-1b5810f62f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:18.790350 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61052e-6846-4569-aaab-1b5810f62f99 None admin] PATH_INFO: `/v3/projects/239afe35723747f6b978e5e6d5aede07` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:18.791474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d61052e-6846-4569-aaab-1b5810f62f99 None admin] RBAC: Authorizing `identity:get_project(project_id=239afe35723747f6b978e5e6d5aede07)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:18.792875 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d61052e-6846-4569-aaab-1b5810f62f99 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:18.794548 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 562/1129] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:48:18 2022] GET /identity/v3/projects/239afe35723747f6b978e5e6d5aede07 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:18.800565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b2d0b-0eb0-46ea-b9f2-ebb78ca4634d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:18.829730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:18.896492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b2d0b-0eb0-46ea-b9f2-ebb78ca4634d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:18.929187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6641ced-e102-4444-84da-229e4a5518be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:18.929791 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6641ced-e102-4444-84da-229e4a5518be None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:18.930109 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6641ced-e102-4444-84da-229e4a5518be None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:18.930380 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6641ced-e102-4444-84da-229e4a5518be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:18.931154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6641ced-e102-4444-84da-229e4a5518be None admin] RBAC: Adding query filter params (role.id=111e800ddb0a411c80664cd51ce13fc6, scope.project.id=239afe35723747f6b978e5e6d5aede07, user.id=b78ed511cb474ce2b31c9b226c5a29ac) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:18.931456 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6641ced-e102-4444-84da-229e4a5518be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:18.932866 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6641ced-e102-4444-84da-229e4a5518be None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:18.939623 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 568/1130] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:48:18 2022] GET /identity/v3/role_assignments?user.id=b78ed511cb474ce2b31c9b226c5a29ac&scope.project.id=239afe35723747f6b978e5e6d5aede07&role.id=111e800ddb0a411c80664cd51ce13fc6 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:20.955244 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c5c64-5c9c-4dbb-a991-ce001d8bb472 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:20.955710 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c5c64-5c9c-4dbb-a991-ce001d8bb472 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:20.955804 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c5c64-5c9c-4dbb-a991-ce001d8bb472 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:20.957133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 563/1131] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:48:20.963048 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8384523-0722-46d5-a61d-9cad40e1100a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:20.963451 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8384523-0722-46d5-a61d-9cad40e1100a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:20.963615 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8384523-0722-46d5-a61d-9cad40e1100a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:20.981390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-b8384523-0722-46d5-a61d-9cad40e1100a None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:48:20.982619 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b8384523-0722-46d5-a61d-9cad40e1100a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:48:21.046657 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 569/1132] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:48:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:48:21.070326 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c5c64-5c9c-4dbb-a991-ce001d8bb472 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:21.125187 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:21.170495 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c5c64-5c9c-4dbb-a991-ce001d8bb472 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:21.202329 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f078b-476d-43f4-a18d-6304b3cdfdd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:21.202889 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f078b-476d-43f4-a18d-6304b3cdfdd2 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:21.203170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f078b-476d-43f4-a18d-6304b3cdfdd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:21.203444 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f078b-476d-43f4-a18d-6304b3cdfdd2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:21.204575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f078b-476d-43f4-a18d-6304b3cdfdd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:21.206013 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f078b-476d-43f4-a18d-6304b3cdfdd2 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:21.208124 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 564/1133] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:48:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:21.214051 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8384523-0722-46d5-a61d-9cad40e1100a None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:21.238387 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:21.312088 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8384523-0722-46d5-a61d-9cad40e1100a None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:21.345331 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfdc2da-11b4-4ec0-9013-15380ca314be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:21.345977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfdc2da-11b4-4ec0-9013-15380ca314be None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:21.346264 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfdc2da-11b4-4ec0-9013-15380ca314be None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:21.346542 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfdc2da-11b4-4ec0-9013-15380ca314be None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:21.347217 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfdc2da-11b4-4ec0-9013-15380ca314be None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:21.349406 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfdc2da-11b4-4ec0-9013-15380ca314be None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:21.437836 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 570/1134] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 2 08:48:21 2022] POST /identity/v3/projects => generated 311 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:48:23.444136 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dda24-6d67-4991-9aa4-5a94dc60b331 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:23.444671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dda24-6d67-4991-9aa4-5a94dc60b331 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:23.444778 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0dda24-6d67-4991-9aa4-5a94dc60b331 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:23.446153 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 565/1135] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:48:23.451575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b5f8e-cfe4-4ae6-a165-20ab4f3c1f7c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:23.451992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b5f8e-cfe4-4ae6-a165-20ab4f3c1f7c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:23.452140 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4b5f8e-cfe4-4ae6-a165-20ab4f3c1f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:23.470519 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-ad4b5f8e-cfe4-4ae6-a165-20ab4f3c1f7c None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:48:23.471696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ad4b5f8e-cfe4-4ae6-a165-20ab4f3c1f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:48:23.535349 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 571/1136] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:48:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:48:23.559810 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0dda24-6d67-4991-9aa4-5a94dc60b331 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:23.659889 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0dda24-6d67-4991-9aa4-5a94dc60b331 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:23.686257 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:23.697736 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480b679-3350-471f-96ed-d576132d9de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:23.698341 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480b679-3350-471f-96ed-d576132d9de7 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:23.698879 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480b679-3350-471f-96ed-d576132d9de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:23.699153 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480b679-3350-471f-96ed-d576132d9de7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:23.700417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4480b679-3350-471f-96ed-d576132d9de7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:23.702856 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4480b679-3350-471f-96ed-d576132d9de7 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:23.704451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 566/1137] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:48:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:48:23.712070 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b5f8e-cfe4-4ae6-a165-20ab4f3c1f7c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:23.721114 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:23.812047 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4b5f8e-cfe4-4ae6-a165-20ab4f3c1f7c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:23.846691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1abf8-04a7-4821-8557-50b1a777cd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:23.847306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1abf8-04a7-4821-8557-50b1a777cd86 None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:23.847608 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1abf8-04a7-4821-8557-50b1a777cd86 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:23.847926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1abf8-04a7-4821-8557-50b1a777cd86 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:23.848596 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c1abf8-04a7-4821-8557-50b1a777cd86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:23.850047 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c1abf8-04a7-4821-8557-50b1a777cd86 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:23.952392 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 572/1138] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 2 08:48:23 2022] POST /identity/v3/users => generated 286 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:48:25.993035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce005da3-5096-4839-8009-bacd51e26e7f None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:25.993516 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce005da3-5096-4839-8009-bacd51e26e7f None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:25.993620 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce005da3-5096-4839-8009-bacd51e26e7f None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:25.994962 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 567/1139] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:48:26.000139 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf7084-bca1-4d8a-9c04-0718b7a84767 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:26.000581 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf7084-bca1-4d8a-9c04-0718b7a84767 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:26.000699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf7084-bca1-4d8a-9c04-0718b7a84767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:26.018328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9ccf7084-bca1-4d8a-9c04-0718b7a84767 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:48:26.019516 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9ccf7084-bca1-4d8a-9c04-0718b7a84767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:48:26.074727 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:26.083739 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 573/1140] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:48:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:48:26.114387 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce005da3-5096-4839-8009-bacd51e26e7f None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:26.212714 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce005da3-5096-4839-8009-bacd51e26e7f None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:26.245558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92df466-dc2b-4164-b870-11d55798bb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:26.246268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92df466-dc2b-4164-b870-11d55798bb9d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:26.246550 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92df466-dc2b-4164-b870-11d55798bb9d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:26.246826 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92df466-dc2b-4164-b870-11d55798bb9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:26.256560 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92df466-dc2b-4164-b870-11d55798bb9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:48:26.261391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-c92df466-dc2b-4164-b870-11d55798bb9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:48:26.261391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:26.261391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:48:26.261391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:48:26.261391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:48:26.261391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:48:26.261391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:48:26.261391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:48:26.261391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:48:26.261391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:48:26.261391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:48:26.261391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:48:26.261391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:48:26.261391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:48:26.261391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:48:26.261391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:48:26.262639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:48:26.262639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:48:26.262639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:48:26.262639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:48:26.262639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:48:26.262639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:48:26.262639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:48:26.262639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:48:26.262639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:48:26.262639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:48:26.262639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:48:26.262639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:48:26.262639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:48:26.262639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:48:26.262639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:48:26.262639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:48:26.263719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:48:26.263719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:48:26.263719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:48:26.263719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:48:26.263719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:48:26.263719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:48:26.263719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:48:26.263719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:48:26.263719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:48:26.263719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:48:26.263719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:48:26.263719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:48:26.263719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 568/1141] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:48:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:48:26.268611 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf7084-bca1-4d8a-9c04-0718b7a84767 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:26.364139 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf7084-bca1-4d8a-9c04-0718b7a84767 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:26.397432 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa61292-8094-45da-9b7a-a23e623326ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:26.398065 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa61292-8094-45da-9b7a-a23e623326ca None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:26.398349 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa61292-8094-45da-9b7a-a23e623326ca None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:26.398629 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa61292-8094-45da-9b7a-a23e623326ca None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:26.399332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa61292-8094-45da-9b7a-a23e623326ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:26.399637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa61292-8094-45da-9b7a-a23e623326ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:26.400993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa61292-8094-45da-9b7a-a23e623326ca None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:26.408786 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 574/1142] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:48:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:26.415149 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92df466-dc2b-4164-b870-11d55798bb9d None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:26.464947 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:26.514159 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92df466-dc2b-4164-b870-11d55798bb9d None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:26.547364 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616eb5d3-d57a-485c-b3f9-4289b9c3749d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:26.547993 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616eb5d3-d57a-485c-b3f9-4289b9c3749d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:26.548277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616eb5d3-d57a-485c-b3f9-4289b9c3749d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:26.548557 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616eb5d3-d57a-485c-b3f9-4289b9c3749d None admin] PATH_INFO: `/v3/users/782c5becd9bb41728ff8faa52141643a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:26.561805 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616eb5d3-d57a-485c-b3f9-4289b9c3749d None admin] RBAC: Authorizing `identity:get_user(user_id=782c5becd9bb41728ff8faa52141643a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:26.563274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616eb5d3-d57a-485c-b3f9-4289b9c3749d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:26.564914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 569/1143] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:48:26 2022] GET /identity/v3/users/782c5becd9bb41728ff8faa52141643a => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:26.570669 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa61292-8094-45da-9b7a-a23e623326ca None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:26.624587 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:26.668072 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa61292-8094-45da-9b7a-a23e623326ca None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:26.699950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31af59-f032-48f8-a445-97fab2886f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:26.700554 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31af59-f032-48f8-a445-97fab2886f76 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:26.700837 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31af59-f032-48f8-a445-97fab2886f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:26.701113 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31af59-f032-48f8-a445-97fab2886f76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:26.767438 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31af59-f032-48f8-a445-97fab2886f76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:26.768820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31af59-f032-48f8-a445-97fab2886f76 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:26.820583 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:26.865161 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 575/1144] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:48:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:48:26.870709 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616eb5d3-d57a-485c-b3f9-4289b9c3749d None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:26.978775 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616eb5d3-d57a-485c-b3f9-4289b9c3749d None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:26.987756 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:27.011879 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6aebf3e-848e-4025-b640-351d11f33759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:27.012508 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aebf3e-848e-4025-b640-351d11f33759 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:27.012787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aebf3e-848e-4025-b640-351d11f33759 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:27.013060 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6aebf3e-848e-4025-b640-351d11f33759 None admin] PATH_INFO: `/v3/projects/2d1e17f92cd64beeb3e8ed122a5a6f27` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:27.014230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aebf3e-848e-4025-b640-351d11f33759 None admin] RBAC: Authorizing `identity:get_project(project_id=2d1e17f92cd64beeb3e8ed122a5a6f27)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:27.015647 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6aebf3e-848e-4025-b640-351d11f33759 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:27.017348 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 570/1145] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:48:26 2022] GET /identity/v3/projects/2d1e17f92cd64beeb3e8ed122a5a6f27 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:27.024342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31af59-f032-48f8-a445-97fab2886f76 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:27.120543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31af59-f032-48f8-a445-97fab2886f76 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:27.153475 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad380a-c261-42a8-b81b-534b277b98e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:27.154113 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad380a-c261-42a8-b81b-534b277b98e4 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:27.154397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad380a-c261-42a8-b81b-534b277b98e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:27.154674 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad380a-c261-42a8-b81b-534b277b98e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:27.155459 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad380a-c261-42a8-b81b-534b277b98e4 None admin] RBAC: Adding query filter params (role.id=8ce0850e15804f3d811e88cc9841e673, scope.project.id=2d1e17f92cd64beeb3e8ed122a5a6f27, user.id=782c5becd9bb41728ff8faa52141643a) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:27.155771 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad380a-c261-42a8-b81b-534b277b98e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:27.157199 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad380a-c261-42a8-b81b-534b277b98e4 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:27.162591 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 576/1146] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:48:27 2022] GET /identity/v3/role_assignments?user.id=782c5becd9bb41728ff8faa52141643a&scope.project.id=2d1e17f92cd64beeb3e8ed122a5a6f27&role.id=8ce0850e15804f3d811e88cc9841e673 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:29.141435 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe0ad0-bdcd-463c-9df2-920692dd2881 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:29.141977 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe0ad0-bdcd-463c-9df2-920692dd2881 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:29.142182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe0ad0-bdcd-463c-9df2-920692dd2881 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:29.143543 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 571/1147] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:48:29.148703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606e02f-73b5-4012-b5dc-e082e46fd343 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:29.149146 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606e02f-73b5-4012-b5dc-e082e46fd343 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:29.149240 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606e02f-73b5-4012-b5dc-e082e46fd343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:29.167472 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-a606e02f-73b5-4012-b5dc-e082e46fd343 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:48:29.168668 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a606e02f-73b5-4012-b5dc-e082e46fd343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:48:29.182049 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:29.234488 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 577/1148] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:48:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:48:29.256832 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe0ad0-bdcd-463c-9df2-920692dd2881 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:29.356143 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe0ad0-bdcd-463c-9df2-920692dd2881 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:29.390547 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1e9b9-ff6c-45f9-a825-5b9b315203a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:29.391166 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1e9b9-ff6c-45f9-a825-5b9b315203a4 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:29.391444 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1e9b9-ff6c-45f9-a825-5b9b315203a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:29.391714 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1e9b9-ff6c-45f9-a825-5b9b315203a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:29.401452 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a1e9b9-ff6c-45f9-a825-5b9b315203a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:48:29.406251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-06a1e9b9-ff6c-45f9-a825-5b9b315203a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:48:29.406251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:29.406251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:48:29.406251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:48:29.406251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:48:29.406251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:48:29.406251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:48:29.406251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:48:29.406251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:48:29.406251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:48:29.406251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:48:29.406251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:48:29.406251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:48:29.406251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:48:29.406251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:48:29.406251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:48:29.407377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:48:29.407377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:48:29.407377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:48:29.407377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:48:29.407377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:48:29.407377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:48:29.407377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:48:29.407377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:48:29.407377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:48:29.407377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:48:29.407377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:48:29.407377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:48:29.407377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:48:29.407377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:48:29.407377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:48:29.407377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:48:29.408451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:48:29.408451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:48:29.408451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:48:29.408451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:48:29.408451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:48:29.408451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:48:29.408451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:48:29.408451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:48:29.408451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:48:29.408451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:48:29.408451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:48:29.408451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:48:29.408451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 572/1149] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:48:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:48:29.413332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606e02f-73b5-4012-b5dc-e082e46fd343 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:29.509776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606e02f-73b5-4012-b5dc-e082e46fd343 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:29.541813 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d6bcf-69fb-41e5-937a-004ba3e6a6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:29.542446 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d6bcf-69fb-41e5-937a-004ba3e6a6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:29.542732 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d6bcf-69fb-41e5-937a-004ba3e6a6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:29.543010 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d6bcf-69fb-41e5-937a-004ba3e6a6a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:29.543789 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d6bcf-69fb-41e5-937a-004ba3e6a6a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:29.544096 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d6bcf-69fb-41e5-937a-004ba3e6a6a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:29.545452 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d6bcf-69fb-41e5-937a-004ba3e6a6a2 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:29.553728 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 578/1150] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:48:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:29.559316 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1e9b9-ff6c-45f9-a825-5b9b315203a4 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:29.594249 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:29.657352 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1e9b9-ff6c-45f9-a825-5b9b315203a4 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:29.689598 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c191047-7b99-49a6-b999-56b7b64041ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:29.690240 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c191047-7b99-49a6-b999-56b7b64041ff None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:29.690520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c191047-7b99-49a6-b999-56b7b64041ff None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:29.690792 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c191047-7b99-49a6-b999-56b7b64041ff None admin] PATH_INFO: `/v3/users/782c5becd9bb41728ff8faa52141643a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:29.691943 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c191047-7b99-49a6-b999-56b7b64041ff None admin] RBAC: Authorizing `identity:get_user(user_id=782c5becd9bb41728ff8faa52141643a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:29.693345 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c191047-7b99-49a6-b999-56b7b64041ff None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:29.694952 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 573/1151] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:48:29 2022] GET /identity/v3/users/782c5becd9bb41728ff8faa52141643a => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:29.700846 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d6bcf-69fb-41e5-937a-004ba3e6a6a2 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:29.713878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:29.804943 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d6bcf-69fb-41e5-937a-004ba3e6a6a2 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:29.838654 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d637b1e-33d8-467f-a255-ff5e5272d48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:29.839288 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d637b1e-33d8-467f-a255-ff5e5272d48c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:29.839572 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d637b1e-33d8-467f-a255-ff5e5272d48c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:29.839850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d637b1e-33d8-467f-a255-ff5e5272d48c None admin] PATH_INFO: `/v3/projects/2d1e17f92cd64beeb3e8ed122a5a6f27` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:29.840994 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d637b1e-33d8-467f-a255-ff5e5272d48c None admin] RBAC: Authorizing `identity:get_project(project_id=2d1e17f92cd64beeb3e8ed122a5a6f27)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:29.842463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d637b1e-33d8-467f-a255-ff5e5272d48c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:29.844108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 579/1152] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:48:29 2022] GET /identity/v3/projects/2d1e17f92cd64beeb3e8ed122a5a6f27 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:29.850226 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c191047-7b99-49a6-b999-56b7b64041ff None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:29.948551 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c191047-7b99-49a6-b999-56b7b64041ff None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:29.951135 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:29.982806 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aaaa44-87d6-40e1-8018-66f4eeddfa45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:29.983471 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aaaa44-87d6-40e1-8018-66f4eeddfa45 None admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:29.983756 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aaaa44-87d6-40e1-8018-66f4eeddfa45 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:29.984033 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aaaa44-87d6-40e1-8018-66f4eeddfa45 None admin] PATH_INFO: `/v3/projects/2d1e17f92cd64beeb3e8ed122a5a6f27/users/782c5becd9bb41728ff8faa52141643a/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:29.986063 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aaaa44-87d6-40e1-8018-66f4eeddfa45 None admin] RBAC: Authorizing `identity:create_grant(project_id=2d1e17f92cd64beeb3e8ed122a5a6f27, user_id=782c5becd9bb41728ff8faa52141643a, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:29.987501 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0aaaa44-87d6-40e1-8018-66f4eeddfa45 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:30.064296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 574/1153] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:48:29 2022] PUT /identity/v3/projects/2d1e17f92cd64beeb3e8ed122a5a6f27/users/782c5becd9bb41728ff8faa52141643a/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:48:32.036339 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4ea22-c0ea-4170-9153-ec7ae8f8abd4 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:32.036846 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4ea22-c0ea-4170-9153-ec7ae8f8abd4 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:32.036933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4ea22-c0ea-4170-9153-ec7ae8f8abd4 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:32.038260 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 580/1154] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:48:32.043287 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f58e7b-96d3-4fee-8928-b188103f3793 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:32.043680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f58e7b-96d3-4fee-8928-b188103f3793 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:32.043839 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f58e7b-96d3-4fee-8928-b188103f3793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:32.061225 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-53f58e7b-96d3-4fee-8928-b188103f3793 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:48:32.062421 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-53f58e7b-96d3-4fee-8928-b188103f3793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:48:32.128065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 575/1155] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:48:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:48:32.151991 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4ea22-c0ea-4170-9153-ec7ae8f8abd4 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:32.228962 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:32.249919 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4ea22-c0ea-4170-9153-ec7ae8f8abd4 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:32.283771 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f6dc9-b960-45f1-89fc-3a2bb9101005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:32.284404 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f6dc9-b960-45f1-89fc-3a2bb9101005 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:32.284691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f6dc9-b960-45f1-89fc-3a2bb9101005 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:32.284983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143f6dc9-b960-45f1-89fc-3a2bb9101005 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:32.295297 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143f6dc9-b960-45f1-89fc-3a2bb9101005 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:48:32.300164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-143f6dc9-b960-45f1-89fc-3a2bb9101005 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:48:32.300164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:32.300164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:48:32.300164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:48:32.300164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:48:32.300164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:48:32.300164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:48:32.300164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:48:32.300164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:48:32.300164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:48:32.300164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:48:32.300164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:48:32.300164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:48:32.300164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:48:32.300164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:48:32.300164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:48:32.301306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:48:32.301306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:48:32.301306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:48:32.301306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:48:32.301306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:48:32.301306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:48:32.301306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:48:32.301306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:48:32.301306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:48:32.301306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:48:32.301306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:48:32.301306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:48:32.301306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:48:32.301306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:48:32.301306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:48:32.301306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:48:32.302246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:48:32.302246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:48:32.302246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:48:32.302246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:48:32.302246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:48:32.302246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:48:32.302246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:48:32.302246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:48:32.302246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:48:32.302246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:48:32.302246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:48:32.302246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:48:32.302246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 581/1156] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:48:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:48:32.307237 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f58e7b-96d3-4fee-8928-b188103f3793 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:32.398465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:32.404989 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f58e7b-96d3-4fee-8928-b188103f3793 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:32.438432 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72cdb4-231d-45bd-a2c1-d33a24fc820e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:32.439035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72cdb4-231d-45bd-a2c1-d33a24fc820e None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:32.439320 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72cdb4-231d-45bd-a2c1-d33a24fc820e None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:32.439597 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae72cdb4-231d-45bd-a2c1-d33a24fc820e None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:32.440306 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae72cdb4-231d-45bd-a2c1-d33a24fc820e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:32.440612 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae72cdb4-231d-45bd-a2c1-d33a24fc820e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:32.442157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae72cdb4-231d-45bd-a2c1-d33a24fc820e None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:32.450161 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 576/1157] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:48:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:32.455879 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f6dc9-b960-45f1-89fc-3a2bb9101005 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:32.555263 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143f6dc9-b960-45f1-89fc-3a2bb9101005 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:32.587847 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33841210-cf11-4a2e-884f-9bb1889001b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:32.588473 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33841210-cf11-4a2e-884f-9bb1889001b6 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:32.588758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33841210-cf11-4a2e-884f-9bb1889001b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:32.589035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33841210-cf11-4a2e-884f-9bb1889001b6 None admin] PATH_INFO: `/v3/users/782c5becd9bb41728ff8faa52141643a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:32.590219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33841210-cf11-4a2e-884f-9bb1889001b6 None admin] RBAC: Authorizing `identity:get_user(user_id=782c5becd9bb41728ff8faa52141643a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:32.591640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33841210-cf11-4a2e-884f-9bb1889001b6 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:32.593209 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 582/1158] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:48:32 2022] GET /identity/v3/users/782c5becd9bb41728ff8faa52141643a => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:32.599533 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72cdb4-231d-45bd-a2c1-d33a24fc820e None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:32.693374 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae72cdb4-231d-45bd-a2c1-d33a24fc820e None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:32.725826 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf1287-e557-462e-95ab-9d6b7d1cadf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:32.726465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf1287-e557-462e-95ab-9d6b7d1cadf3 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:32.726725 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf1287-e557-462e-95ab-9d6b7d1cadf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:32.726995 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf1287-e557-462e-95ab-9d6b7d1cadf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:32.749810 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:32.791206 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf1287-e557-462e-95ab-9d6b7d1cadf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:32.792623 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf1287-e557-462e-95ab-9d6b7d1cadf3 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:32.889169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 577/1159] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:48:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:48:32.895763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33841210-cf11-4a2e-884f-9bb1889001b6 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:32.904944 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:32.993003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33841210-cf11-4a2e-884f-9bb1889001b6 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:33.025263 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d68da26-9c81-4ade-b487-258ce2ef638c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:33.025914 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68da26-9c81-4ade-b487-258ce2ef638c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:33.026196 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68da26-9c81-4ade-b487-258ce2ef638c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:33.026471 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68da26-9c81-4ade-b487-258ce2ef638c None admin] PATH_INFO: `/v3/projects/2d1e17f92cd64beeb3e8ed122a5a6f27` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:33.027597 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d68da26-9c81-4ade-b487-258ce2ef638c None admin] RBAC: Authorizing `identity:get_project(project_id=2d1e17f92cd64beeb3e8ed122a5a6f27)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:33.029012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d68da26-9c81-4ade-b487-258ce2ef638c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:33.030661 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 583/1160] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:48:32 2022] GET /identity/v3/projects/2d1e17f92cd64beeb3e8ed122a5a6f27 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:33.036776 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf1287-e557-462e-95ab-9d6b7d1cadf3 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:33.090234 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:33.133377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf1287-e557-462e-95ab-9d6b7d1cadf3 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:33.166221 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653707fa-749c-4c47-91e6-00a1a6e96438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:33.166836 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653707fa-749c-4c47-91e6-00a1a6e96438 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:33.167117 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653707fa-749c-4c47-91e6-00a1a6e96438 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:33.167395 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653707fa-749c-4c47-91e6-00a1a6e96438 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:33.168175 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653707fa-749c-4c47-91e6-00a1a6e96438 None admin] RBAC: Adding query filter params (role.id=8ce0850e15804f3d811e88cc9841e673, scope.project.id=2d1e17f92cd64beeb3e8ed122a5a6f27, user.id=782c5becd9bb41728ff8faa52141643a) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:33.168490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653707fa-749c-4c47-91e6-00a1a6e96438 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:33.169932 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653707fa-749c-4c47-91e6-00a1a6e96438 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:33.176082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 578/1161] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:48:33 2022] GET /identity/v3/role_assignments?user.id=782c5becd9bb41728ff8faa52141643a&scope.project.id=2d1e17f92cd64beeb3e8ed122a5a6f27&role.id=8ce0850e15804f3d811e88cc9841e673 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:35.174081 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4f52d-ccf2-462a-9674-bb377530c427 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:35.174662 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4f52d-ccf2-462a-9674-bb377530c427 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:35.174662 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf4f52d-ccf2-462a-9674-bb377530c427 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:35.176695 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 584/1162] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:48:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:48:35.181929 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b6fd6-5fc0-4568-b077-08d828a9f98f None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:35.182320 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b6fd6-5fc0-4568-b077-08d828a9f98f None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:35.182496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b6fd6-5fc0-4568-b077-08d828a9f98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:35.200472 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-2f9b6fd6-5fc0-4568-b077-08d828a9f98f None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:48:35.201587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-2f9b6fd6-5fc0-4568-b077-08d828a9f98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:48:35.266264 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 579/1163] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:48:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:48:35.288826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4f52d-ccf2-462a-9674-bb377530c427 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:35.362850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:35.388003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf4f52d-ccf2-462a-9674-bb377530c427 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:35.422485 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bfebf-5283-447a-aabb-e8f7438a8078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:35.423105 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bfebf-5283-447a-aabb-e8f7438a8078 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:35.423390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bfebf-5283-447a-aabb-e8f7438a8078 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:35.423671 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19bfebf-5283-447a-aabb-e8f7438a8078 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:35.489617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19bfebf-5283-447a-aabb-e8f7438a8078 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:35.491015 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19bfebf-5283-447a-aabb-e8f7438a8078 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:35.560359 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:35.591378 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 585/1164] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:48:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:48:35.598913 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b6fd6-5fc0-4568-b077-08d828a9f98f None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:35.647514 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:35.699912 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b6fd6-5fc0-4568-b077-08d828a9f98f None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:35.734029 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da092cb9-4615-4ea8-92a9-ab5c416aa1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:35.734665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da092cb9-4615-4ea8-92a9-ab5c416aa1ff None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:35.734949 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da092cb9-4615-4ea8-92a9-ab5c416aa1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:35.735224 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da092cb9-4615-4ea8-92a9-ab5c416aa1ff None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:35.742082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da092cb9-4615-4ea8-92a9-ab5c416aa1ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:35.743464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da092cb9-4615-4ea8-92a9-ab5c416aa1ff None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:35.755277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-da092cb9-4615-4ea8-92a9-ab5c416aa1ff None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 02 08:48:35.755277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:35.755277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 02 08:48:35.755277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 02 08:48:35.755277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:48:35.755277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:48:35.755277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:48:35.755277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:48:35.755277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 02 08:48:35.755277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:48:35.755277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:48:35.755277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:48:35.755277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:35.755277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:48:35.755277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:48:35.755277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:48:35.755277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:48:35.756520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:48:35.756520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:48:35.756520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:48:35.756520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:48:35.756520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:48:35.756520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:48:35.756520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:48:35.756520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:48:35.756520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:48:35.756520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:48:35.756520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:48:35.756520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:48:35.756520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:48:35.756520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:48:35.756520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:48:35.756520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:48:35.757652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:48:35.757652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:48:35.757652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:48:35.757652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:48:35.757652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:48:35.757652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:48:35.757652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:48:35.757652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:48:35.757652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:48:35.757652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:48:35.757652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 02 08:48:35.757652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:48:35.757652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 02 08:48:35.757652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:48:35.757652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 580/1165] 10.222.0.46 () {56 vars in 1117 bytes} [Tue Aug 2 08:48:35 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:48:35.762592 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bfebf-5283-447a-aabb-e8f7438a8078 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:35.863360 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19bfebf-5283-447a-aabb-e8f7438a8078 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:35.897336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee65c83-4479-440d-a486-fa55ae75207b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:35.897987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee65c83-4479-440d-a486-fa55ae75207b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:35.898274 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee65c83-4479-440d-a486-fa55ae75207b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:35.898549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee65c83-4479-440d-a486-fa55ae75207b None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:35.899245 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee65c83-4479-440d-a486-fa55ae75207b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:35.900403 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee65c83-4479-440d-a486-fa55ae75207b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:35.900914 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee65c83-4479-440d-a486-fa55ae75207b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:35.908274 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 586/1166] 10.222.0.46 () {56 vars in 1115 bytes} [Tue Aug 2 08:48:35 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:37.890429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93e61e-5319-4902-b008-0725fbf3e18d None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:37.890960 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93e61e-5319-4902-b008-0725fbf3e18d None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:37.890960 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e93e61e-5319-4902-b008-0725fbf3e18d None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:37.892339 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 581/1167] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:48:37.898356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d5cea-9d01-4cc4-a1b7-4c8e963f6bc1 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:37.898742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d5cea-9d01-4cc4-a1b7-4c8e963f6bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:37.898919 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d5cea-9d01-4cc4-a1b7-4c8e963f6bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:37.917279 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-bc9d5cea-9d01-4cc4-a1b7-4c8e963f6bc1 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:48:37.918496 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bc9d5cea-9d01-4cc4-a1b7-4c8e963f6bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:48:37.949073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:37.986096 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 587/1168] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:48:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:48:38.009125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93e61e-5319-4902-b008-0725fbf3e18d None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:38.107423 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e93e61e-5319-4902-b008-0725fbf3e18d None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:38.118355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:38.140846 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc88507-0443-42bb-9342-25751570234c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:38.141390 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc88507-0443-42bb-9342-25751570234c None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:38.141672 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc88507-0443-42bb-9342-25751570234c None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:38.141976 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc88507-0443-42bb-9342-25751570234c None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:38.142691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc88507-0443-42bb-9342-25751570234c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:38.144248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc88507-0443-42bb-9342-25751570234c None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:38.226711 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 582/1169] 10.222.0.46 () {60 vars in 1138 bytes} [Tue Aug 2 08:48:38 2022] POST /identity/v3/domains => generated 267 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:48:40.251093 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e3110-7bd4-4f62-ab2c-8131d16c001b None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:40.251614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e3110-7bd4-4f62-ab2c-8131d16c001b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:40.251614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e3110-7bd4-4f62-ab2c-8131d16c001b None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:40.252900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 588/1170] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:48:40.258567 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af74b80-68d4-4379-962e-386b8f52b61a None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:40.258988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af74b80-68d4-4379-962e-386b8f52b61a None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:40.259118 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af74b80-68d4-4379-962e-386b8f52b61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:40.277515 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-4af74b80-68d4-4379-962e-386b8f52b61a None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:48:40.278710 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4af74b80-68d4-4379-962e-386b8f52b61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:48:40.343044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 583/1171] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:48:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:48:40.366709 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e3110-7bd4-4f62-ab2c-8131d16c001b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:40.466509 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7e3110-7bd4-4f62-ab2c-8131d16c001b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:40.499559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d8b636-53a4-4265-bc74-60f90005d335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:40.500187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8b636-53a4-4265-bc74-60f90005d335 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:40.500468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8b636-53a4-4265-bc74-60f90005d335 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:40.500740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8b636-53a4-4265-bc74-60f90005d335 None admin] PATH_INFO: `/v3/domains/07085fa5151e4b248a2abb12ce631cb8` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:40.510996 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d8b636-53a4-4265-bc74-60f90005d335 None admin] RBAC: Authorizing `identity:get_domain(domain_id=07085fa5151e4b248a2abb12ce631cb8)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:40.512370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d8b636-53a4-4265-bc74-60f90005d335 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:40.513998 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 589/1172] 10.222.0.46 () {56 vars in 1183 bytes} [Tue Aug 2 08:48:40 2022] GET /identity/v3/domains/07085fa5151e4b248a2abb12ce631cb8 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:40.519849 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af74b80-68d4-4379-962e-386b8f52b61a None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:40.612661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:40.616096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af74b80-68d4-4379-962e-386b8f52b61a None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:40.649817 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95c2b37-9da4-43c9-9c6e-e383d6cc1480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:40.650456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c2b37-9da4-43c9-9c6e-e383d6cc1480 None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:40.650741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c2b37-9da4-43c9-9c6e-e383d6cc1480 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:40.651019 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95c2b37-9da4-43c9-9c6e-e383d6cc1480 None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:40.651691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95c2b37-9da4-43c9-9c6e-e383d6cc1480 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:40.653151 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95c2b37-9da4-43c9-9c6e-e383d6cc1480 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:40.733042 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 584/1173] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 2 08:48:40 2022] POST /identity/v3/projects => generated 361 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:48:42.762420 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbc1d8-0855-4ef1-b0d3-acad1a096222 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:42.762949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbc1d8-0855-4ef1-b0d3-acad1a096222 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:42.763064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbc1d8-0855-4ef1-b0d3-acad1a096222 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:42.764323 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 590/1174] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:48:42.769557 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8df836-78f4-45a3-8e9c-d71937d84dde None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:42.769966 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8df836-78f4-45a3-8e9c-d71937d84dde None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:42.770152 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8df836-78f4-45a3-8e9c-d71937d84dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:42.788351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-5f8df836-78f4-45a3-8e9c-d71937d84dde None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:48:42.789637 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-5f8df836-78f4-45a3-8e9c-d71937d84dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:48:42.855738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 585/1175] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:48:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:48:42.880204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbc1d8-0855-4ef1-b0d3-acad1a096222 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:42.911106 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:42.983292 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbc1d8-0855-4ef1-b0d3-acad1a096222 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:43.016284 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64388891-a78e-48c4-ba08-ab32c59f1d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:43.016862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64388891-a78e-48c4-ba08-ab32c59f1d8b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:43.017144 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64388891-a78e-48c4-ba08-ab32c59f1d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:43.017419 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64388891-a78e-48c4-ba08-ab32c59f1d8b None admin] PATH_INFO: `/v3/domains/07085fa5151e4b248a2abb12ce631cb8` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:43.018578 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64388891-a78e-48c4-ba08-ab32c59f1d8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=07085fa5151e4b248a2abb12ce631cb8)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:43.020526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64388891-a78e-48c4-ba08-ab32c59f1d8b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:43.022125 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 591/1176] 10.222.0.46 () {56 vars in 1183 bytes} [Tue Aug 2 08:48:42 2022] GET /identity/v3/domains/07085fa5151e4b248a2abb12ce631cb8 => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:43.028067 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8df836-78f4-45a3-8e9c-d71937d84dde None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:43.106406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:43.123599 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8df836-78f4-45a3-8e9c-d71937d84dde None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:43.156566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1084bc4-238c-44de-a889-d13beba0dfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:43.157171 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1084bc4-238c-44de-a889-d13beba0dfa1 None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:43.157455 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1084bc4-238c-44de-a889-d13beba0dfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:43.157732 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1084bc4-238c-44de-a889-d13beba0dfa1 None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:43.158445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1084bc4-238c-44de-a889-d13beba0dfa1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:43.160459 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1084bc4-238c-44de-a889-d13beba0dfa1 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:43.242274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 586/1177] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 2 08:48:43 2022] POST /identity/v3/users => generated 311 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:48:45.281610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221a9d5-7273-410f-8f34-9f7e41d56619 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:45.282131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221a9d5-7273-410f-8f34-9f7e41d56619 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:45.282131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7221a9d5-7273-410f-8f34-9f7e41d56619 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:45.283456 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 592/1178] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:48:45.289596 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f968d0-5cd6-4630-8da8-1c4f14ea59ae None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:45.290033 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f968d0-5cd6-4630-8da8-1c4f14ea59ae None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:45.290186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f968d0-5cd6-4630-8da8-1c4f14ea59ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:45.307822 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-b3f968d0-5cd6-4630-8da8-1c4f14ea59ae None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:48:45.309009 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-b3f968d0-5cd6-4630-8da8-1c4f14ea59ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:48:45.373169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 587/1179] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:48:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:48:45.396744 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221a9d5-7273-410f-8f34-9f7e41d56619 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:45.487822 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:45.498950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7221a9d5-7273-410f-8f34-9f7e41d56619 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:45.533742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8a4fb-589e-4461-bcbc-a105a28898a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:45.534347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8a4fb-589e-4461-bcbc-a105a28898a3 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:45.534654 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8a4fb-589e-4461-bcbc-a105a28898a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:45.534934 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e8a4fb-589e-4461-bcbc-a105a28898a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:45.544802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e8a4fb-589e-4461-bcbc-a105a28898a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:48:45.549650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-b3e8a4fb-589e-4461-bcbc-a105a28898a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:48:45.549650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:45.549650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:48:45.549650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:48:45.549650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:48:45.549650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:48:45.549650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:48:45.549650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:48:45.549650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:48:45.549650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:48:45.549650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:48:45.549650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:48:45.549650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:48:45.549650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:48:45.549650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:48:45.549650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:48:45.551039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:48:45.551039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:48:45.551039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:48:45.551039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:48:45.551039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:48:45.551039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:48:45.551039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:48:45.551039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:48:45.551039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:48:45.551039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:48:45.551039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:48:45.551039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:48:45.551039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:48:45.551039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:48:45.551039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:48:45.551039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:48:45.552108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:48:45.552108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:48:45.552108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:48:45.552108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:48:45.552108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:48:45.552108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:48:45.552108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:48:45.552108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:48:45.552108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:48:45.552108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:48:45.552108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:48:45.552108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:48:45.552108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 593/1180] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:48:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:48:45.557159 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f968d0-5cd6-4630-8da8-1c4f14ea59ae None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:45.622102 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:45.655524 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f968d0-5cd6-4630-8da8-1c4f14ea59ae None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:45.688261 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28638bfa-f788-4700-953d-d226383bc259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:45.688871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28638bfa-f788-4700-953d-d226383bc259 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:45.689154 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28638bfa-f788-4700-953d-d226383bc259 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:45.689430 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28638bfa-f788-4700-953d-d226383bc259 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:45.690165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28638bfa-f788-4700-953d-d226383bc259 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:45.690480 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28638bfa-f788-4700-953d-d226383bc259 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:45.692367 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28638bfa-f788-4700-953d-d226383bc259 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:45.700448 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 588/1181] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:48:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:45.706307 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8a4fb-589e-4461-bcbc-a105a28898a3 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:45.803961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e8a4fb-589e-4461-bcbc-a105a28898a3 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:45.837331 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349d39e-b5b7-46b3-9259-ac6ef10980ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:45.837986 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349d39e-b5b7-46b3-9259-ac6ef10980ab None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:45.838273 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349d39e-b5b7-46b3-9259-ac6ef10980ab None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:45.838551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e349d39e-b5b7-46b3-9259-ac6ef10980ab None admin] PATH_INFO: `/v3/users/e510c39196774458b6542af7b150d663` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:45.886442 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349d39e-b5b7-46b3-9259-ac6ef10980ab None admin] RBAC: Authorizing `identity:get_user(user_id=e510c39196774458b6542af7b150d663)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:45.887947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e349d39e-b5b7-46b3-9259-ac6ef10980ab None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:45.889574 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 594/1182] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:48:45 2022] GET /identity/v3/users/e510c39196774458b6542af7b150d663 => generated 311 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:45.895663 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28638bfa-f788-4700-953d-d226383bc259 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:45.993515 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28638bfa-f788-4700-953d-d226383bc259 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:46.020350 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:46.027186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec094b-e1be-4a14-8a49-afd2356029fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:46.048441 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec094b-e1be-4a14-8a49-afd2356029fd None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:46.048441 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec094b-e1be-4a14-8a49-afd2356029fd None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:46.048441 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ec094b-e1be-4a14-8a49-afd2356029fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:46.098893 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec094b-e1be-4a14-8a49-afd2356029fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:46.100243 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ec094b-e1be-4a14-8a49-afd2356029fd None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:46.194484 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 589/1183] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:48:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 08:48:46.200690 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349d39e-b5b7-46b3-9259-ac6ef10980ab None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:46.216465 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:46.298115 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e349d39e-b5b7-46b3-9259-ac6ef10980ab None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:46.332164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e7ce6-daea-4e8b-bd53-211ef38990f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:46.332795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e7ce6-daea-4e8b-bd53-211ef38990f5 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:46.333077 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e7ce6-daea-4e8b-bd53-211ef38990f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:46.333354 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e7ce6-daea-4e8b-bd53-211ef38990f5 None admin] PATH_INFO: `/v3/projects/c52a57f6cb714cd0b085183b1d78644b` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:46.334512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392e7ce6-daea-4e8b-bd53-211ef38990f5 None admin] RBAC: Authorizing `identity:get_project(project_id=c52a57f6cb714cd0b085183b1d78644b)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:46.335935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392e7ce6-daea-4e8b-bd53-211ef38990f5 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:46.337565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 595/1184] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:48:46 2022] GET /identity/v3/projects/c52a57f6cb714cd0b085183b1d78644b => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:46.343587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec094b-e1be-4a14-8a49-afd2356029fd None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:46.372092 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:46.439868 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ec094b-e1be-4a14-8a49-afd2356029fd None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:46.473448 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197907e-358c-4c18-ad14-7034bebf86b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:46.474081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197907e-358c-4c18-ad14-7034bebf86b0 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:46.474366 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197907e-358c-4c18-ad14-7034bebf86b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:46.474643 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197907e-358c-4c18-ad14-7034bebf86b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:46.475423 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197907e-358c-4c18-ad14-7034bebf86b0 None admin] RBAC: Adding query filter params (role.id=8ce0850e15804f3d811e88cc9841e673, scope.project.id=c52a57f6cb714cd0b085183b1d78644b, user.id=e510c39196774458b6542af7b150d663) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:46.475735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197907e-358c-4c18-ad14-7034bebf86b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:46.477189 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197907e-358c-4c18-ad14-7034bebf86b0 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:46.482459 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 590/1185] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:48:46 2022] GET /identity/v3/role_assignments?user.id=e510c39196774458b6542af7b150d663&scope.project.id=c52a57f6cb714cd0b085183b1d78644b&role.id=8ce0850e15804f3d811e88cc9841e673 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:48.434967 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c8ae4-371b-4cb3-a926-5ed4b965b59b None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:48.435516 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c8ae4-371b-4cb3-a926-5ed4b965b59b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:48.435516 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159c8ae4-371b-4cb3-a926-5ed4b965b59b None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:48.436854 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 596/1186] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:48:48.442198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4613715-c565-429a-8dac-f583b5f10670 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:48.442614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4613715-c565-429a-8dac-f583b5f10670 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:48.442762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4613715-c565-429a-8dac-f583b5f10670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:48.461367 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-d4613715-c565-429a-8dac-f583b5f10670 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:48:48.462585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-d4613715-c565-429a-8dac-f583b5f10670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:48:48.527758 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 591/1187] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:48:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:48:48.549982 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159c8ae4-371b-4cb3-a926-5ed4b965b59b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:48.630931 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:48.648869 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159c8ae4-371b-4cb3-a926-5ed4b965b59b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:48.682566 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795c9ca-12bf-482f-8ff9-47f39c25f65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:48.683197 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795c9ca-12bf-482f-8ff9-47f39c25f65c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:48.683480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795c9ca-12bf-482f-8ff9-47f39c25f65c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:48.683755 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795c9ca-12bf-482f-8ff9-47f39c25f65c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:48.693517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8795c9ca-12bf-482f-8ff9-47f39c25f65c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:48:48.698315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-8795c9ca-12bf-482f-8ff9-47f39c25f65c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:48:48.698315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:48.698315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:48:48.698315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:48:48.698315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:48:48.698315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:48:48.698315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:48:48.698315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:48:48.698315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:48:48.698315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:48:48.698315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:48:48.698315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:48:48.698315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:48:48.698315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:48:48.698315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:48:48.698315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:48:48.699729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:48:48.699729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:48:48.699729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:48:48.699729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:48:48.699729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:48:48.699729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:48:48.699729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:48:48.699729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:48:48.699729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:48:48.699729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:48:48.699729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:48:48.699729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:48:48.699729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:48:48.699729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:48:48.699729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:48:48.699729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:48:48.700839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:48:48.700839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:48:48.700839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:48:48.700839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:48:48.700839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:48:48.700839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:48:48.700839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:48:48.700839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:48:48.700839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:48:48.700839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:48:48.700839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:48:48.700839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:48:48.700839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 597/1188] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:48:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:48:48.705707 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4613715-c565-429a-8dac-f583b5f10670 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:48.731787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:48.804717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4613715-c565-429a-8dac-f583b5f10670 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:48.837708 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f887a918-e348-4afb-8788-46ad46a35674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:48.838343 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f887a918-e348-4afb-8788-46ad46a35674 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:48.838631 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f887a918-e348-4afb-8788-46ad46a35674 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:48.838909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f887a918-e348-4afb-8788-46ad46a35674 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:48.839614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f887a918-e348-4afb-8788-46ad46a35674 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:48.839920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f887a918-e348-4afb-8788-46ad46a35674 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:48.841275 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f887a918-e348-4afb-8788-46ad46a35674 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:48.849854 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 592/1189] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:48:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:48.855767 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795c9ca-12bf-482f-8ff9-47f39c25f65c None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:48.955774 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795c9ca-12bf-482f-8ff9-47f39c25f65c None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:48.988627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2ea16-aeeb-42bf-8c8d-f7e855ce7fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:48.989261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2ea16-aeeb-42bf-8c8d-f7e855ce7fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:48.989545 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2ea16-aeeb-42bf-8c8d-f7e855ce7fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:48.989825 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2ea16-aeeb-42bf-8c8d-f7e855ce7fb0 None admin] PATH_INFO: `/v3/users/e510c39196774458b6542af7b150d663` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:48.991001 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2ea16-aeeb-42bf-8c8d-f7e855ce7fb0 None admin] RBAC: Authorizing `identity:get_user(user_id=e510c39196774458b6542af7b150d663)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:48.992431 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2ea16-aeeb-42bf-8c8d-f7e855ce7fb0 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:48.994043 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 598/1190] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:48:48 2022] GET /identity/v3/users/e510c39196774458b6542af7b150d663 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:48.999816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f887a918-e348-4afb-8788-46ad46a35674 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:49.083134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:49.098190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f887a918-e348-4afb-8788-46ad46a35674 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:49.132339 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12510dfe-b580-45ab-94af-bdbdd5af1975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:49.132973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12510dfe-b580-45ab-94af-bdbdd5af1975 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:49.133258 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12510dfe-b580-45ab-94af-bdbdd5af1975 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:49.133537 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12510dfe-b580-45ab-94af-bdbdd5af1975 None admin] PATH_INFO: `/v3/projects/c52a57f6cb714cd0b085183b1d78644b` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:49.134710 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12510dfe-b580-45ab-94af-bdbdd5af1975 None admin] RBAC: Authorizing `identity:get_project(project_id=c52a57f6cb714cd0b085183b1d78644b)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:49.136146 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12510dfe-b580-45ab-94af-bdbdd5af1975 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:49.137852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 593/1191] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:48:48 2022] GET /identity/v3/projects/c52a57f6cb714cd0b085183b1d78644b => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:49.143807 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2ea16-aeeb-42bf-8c8d-f7e855ce7fb0 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:49.156852 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:49.241785 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2ea16-aeeb-42bf-8c8d-f7e855ce7fb0 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:49.274733 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8fd50-d1fb-493a-b6bf-56a6e8c4fd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:49.275398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8fd50-d1fb-493a-b6bf-56a6e8c4fd7a None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:49.275678 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8fd50-d1fb-493a-b6bf-56a6e8c4fd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:49.275956 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8fd50-d1fb-493a-b6bf-56a6e8c4fd7a None admin] PATH_INFO: `/v3/projects/c52a57f6cb714cd0b085183b1d78644b/users/e510c39196774458b6542af7b150d663/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:49.277877 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8fd50-d1fb-493a-b6bf-56a6e8c4fd7a None admin] RBAC: Authorizing `identity:create_grant(project_id=c52a57f6cb714cd0b085183b1d78644b, user_id=e510c39196774458b6542af7b150d663, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:49.279329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd8fd50-d1fb-493a-b6bf-56a6e8c4fd7a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:49.350371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 599/1192] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:48:49 2022] PUT /identity/v3/projects/c52a57f6cb714cd0b085183b1d78644b/users/e510c39196774458b6542af7b150d663/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:48:51.336169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d78d9-d68c-422d-b5eb-f14986d6847b None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:51.336668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d78d9-d68c-422d-b5eb-f14986d6847b None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:51.336668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d78d9-d68c-422d-b5eb-f14986d6847b None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:51.338035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 594/1193] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:48:51.343319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc98eb-5df4-4064-a421-36e3033f5cd2 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:51.343742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc98eb-5df4-4064-a421-36e3033f5cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:51.343883 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcc98eb-5df4-4064-a421-36e3033f5cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:51.361481 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-1fcc98eb-5df4-4064-a421-36e3033f5cd2 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:48:51.363225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1fcc98eb-5df4-4064-a421-36e3033f5cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:48:51.422766 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:51.427679 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 600/1194] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:48:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:48:51.451407 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d78d9-d68c-422d-b5eb-f14986d6847b None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:51.551471 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d78d9-d68c-422d-b5eb-f14986d6847b None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:51.583662 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde807c8-c043-459d-b24b-085a7d1a9fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:51.584283 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde807c8-c043-459d-b24b-085a7d1a9fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:51.584563 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde807c8-c043-459d-b24b-085a7d1a9fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:51.584837 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde807c8-c043-459d-b24b-085a7d1a9fe9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:51.594658 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde807c8-c043-459d-b24b-085a7d1a9fe9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:48:51.599974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-fde807c8-c043-459d-b24b-085a7d1a9fe9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:48:51.599974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:48:51.599974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:48:51.599974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:48:51.599974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:48:51.599974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:48:51.599974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:48:51.599974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:48:51.599974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:48:51.599974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:48:51.599974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:48:51.599974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:48:51.599974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:48:51.599974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:48:51.599974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:48:51.599974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:48:51.601109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:48:51.601109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:48:51.601109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:48:51.601109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:48:51.601109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:48:51.601109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:48:51.601109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:48:51.601109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:48:51.601109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:48:51.601109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:48:51.601109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:48:51.601109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:48:51.601109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:48:51.601109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:48:51.601109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:48:51.601109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:48:51.602523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:48:51.602523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:48:51.602523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:48:51.602523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:48:51.602523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:48:51.602523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:48:51.602523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:48:51.602523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:48:51.602523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:48:51.602523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:48:51.602523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:48:51.602523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:48:51.602523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 595/1195] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:48:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:48:51.607089 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc98eb-5df4-4064-a421-36e3033f5cd2 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:51.703903 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcc98eb-5df4-4064-a421-36e3033f5cd2 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:51.736639 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63c5e6-b948-444a-8998-ec0bd0b93416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:51.737248 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63c5e6-b948-444a-8998-ec0bd0b93416 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:51.737529 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63c5e6-b948-444a-8998-ec0bd0b93416 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:51.738482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63c5e6-b948-444a-8998-ec0bd0b93416 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:51.739208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d63c5e6-b948-444a-8998-ec0bd0b93416 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:51.739519 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d63c5e6-b948-444a-8998-ec0bd0b93416 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:51.740876 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d63c5e6-b948-444a-8998-ec0bd0b93416 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:51.748594 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 601/1196] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:48:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:51.754353 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde807c8-c043-459d-b24b-085a7d1a9fe9 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:51.770783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:51.854458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde807c8-c043-459d-b24b-085a7d1a9fe9 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:51.887702 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99e3f2-9c8c-4045-9bfc-3c47a2e1f0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:51.888323 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99e3f2-9c8c-4045-9bfc-3c47a2e1f0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:51.888600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99e3f2-9c8c-4045-9bfc-3c47a2e1f0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:51.888866 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99e3f2-9c8c-4045-9bfc-3c47a2e1f0d1 None admin] PATH_INFO: `/v3/users/e510c39196774458b6542af7b150d663` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:51.890087 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99e3f2-9c8c-4045-9bfc-3c47a2e1f0d1 None admin] RBAC: Authorizing `identity:get_user(user_id=e510c39196774458b6542af7b150d663)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:51.891495 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99e3f2-9c8c-4045-9bfc-3c47a2e1f0d1 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:51.893135 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 596/1197] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 2 08:48:51 2022] GET /identity/v3/users/e510c39196774458b6542af7b150d663 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:51.899494 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63c5e6-b948-444a-8998-ec0bd0b93416 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:51.959850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:51.999350 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63c5e6-b948-444a-8998-ec0bd0b93416 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:52.031716 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0653ba8-9686-4472-a9b9-9dbb830bd00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:52.032328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0653ba8-9686-4472-a9b9-9dbb830bd00d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:52.032612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0653ba8-9686-4472-a9b9-9dbb830bd00d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:52.032888 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0653ba8-9686-4472-a9b9-9dbb830bd00d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:52.098702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0653ba8-9686-4472-a9b9-9dbb830bd00d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:52.100148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0653ba8-9686-4472-a9b9-9dbb830bd00d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:52.156773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:52.196436 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 602/1198] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:48:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:48:52.202625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99e3f2-9c8c-4045-9bfc-3c47a2e1f0d1 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:52.281417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:48:52.298543 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99e3f2-9c8c-4045-9bfc-3c47a2e1f0d1 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:52.331459 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e9b42-5025-4048-8a5e-31ae12b8b66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:52.332573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e9b42-5025-4048-8a5e-31ae12b8b66d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:52.332849 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e9b42-5025-4048-8a5e-31ae12b8b66d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:52.333118 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e9b42-5025-4048-8a5e-31ae12b8b66d None admin] PATH_INFO: `/v3/projects/c52a57f6cb714cd0b085183b1d78644b` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:52.334266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e9b42-5025-4048-8a5e-31ae12b8b66d None admin] RBAC: Authorizing `identity:get_project(project_id=c52a57f6cb714cd0b085183b1d78644b)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:52.335666 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e9b42-5025-4048-8a5e-31ae12b8b66d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:52.337325 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 597/1199] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 2 08:48:52 2022] GET /identity/v3/projects/c52a57f6cb714cd0b085183b1d78644b => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:48:52.343291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0653ba8-9686-4472-a9b9-9dbb830bd00d None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:48:52.439035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0653ba8-9686-4472-a9b9-9dbb830bd00d None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:48:52.471351 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d695645-47c5-4254-8776-e41683927183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:48:52.472451 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d695645-47c5-4254-8776-e41683927183 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:48:52.472744 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d695645-47c5-4254-8776-e41683927183 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:48:52.473019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d695645-47c5-4254-8776-e41683927183 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:48:52.473801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d695645-47c5-4254-8776-e41683927183 None admin] RBAC: Adding query filter params (role.id=8ce0850e15804f3d811e88cc9841e673, scope.project.id=c52a57f6cb714cd0b085183b1d78644b, user.id=e510c39196774458b6542af7b150d663) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:48:52.474139 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d695645-47c5-4254-8776-e41683927183 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:48:52.475559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d695645-47c5-4254-8776-e41683927183 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:48:52.481197 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 603/1200] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:48:52 2022] GET /identity/v3/role_assignments?user.id=e510c39196774458b6542af7b150d663&scope.project.id=c52a57f6cb714cd0b085183b1d78644b&role.id=8ce0850e15804f3d811e88cc9841e673 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:49:02.327570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=25 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:49:02.478670 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=87 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:05.025825 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1848007-063f-44db-9308-75486cc13150 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:05.026439 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1848007-063f-44db-9308-75486cc13150 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:05.026439 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1848007-063f-44db-9308-75486cc13150 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:05.027748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 598/1201] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:50:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:50:05.032868 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2bc3d-6670-429b-9eef-58fde3adacb5 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:05.037030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2bc3d-6670-429b-9eef-58fde3adacb5 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:05.037030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef2bc3d-6670-429b-9eef-58fde3adacb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:05.051104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-8ef2bc3d-6670-429b-9eef-58fde3adacb5 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:50:05.052299 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8ef2bc3d-6670-429b-9eef-58fde3adacb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:50:05.118406 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 604/1202] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:50:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:50:05.143408 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1848007-063f-44db-9308-75486cc13150 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:05.243536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1848007-063f-44db-9308-75486cc13150 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:05.276798 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0ed58e-6a89-4d20-8950-4fe2e87ce8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:05.277425 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ed58e-6a89-4d20-8950-4fe2e87ce8cd None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:05.277704 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ed58e-6a89-4d20-8950-4fe2e87ce8cd None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:05.278008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ed58e-6a89-4d20-8950-4fe2e87ce8cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:05.288037 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0ed58e-6a89-4d20-8950-4fe2e87ce8cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:05.289518 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0ed58e-6a89-4d20-8950-4fe2e87ce8cd None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:05.303762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-0e0ed58e-6a89-4d20-8950-4fe2e87ce8cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:05.303762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:05.303762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:05.303762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:05.303762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:05.303762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:05.303762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:05.303762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:05.303762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:05.303762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:05.303762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:05.303762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:05.303762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:50:05.303762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:50:05.303762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:50:05.305429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:50:05.305429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:05.305429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:05.305429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:05.305429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:05.305429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:05.305429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:05.305429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:05.305429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:05.305429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:05.305429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:05.305429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:05.305429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:05.305429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:05.305429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:05.305429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:05.305429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:05.306923 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:50:05.306923 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:50:05.306923 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:05.306923 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:50:05.306923 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 599/1203] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:50:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:05.311408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2bc3d-6670-429b-9eef-58fde3adacb5 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:05.408124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef2bc3d-6670-429b-9eef-58fde3adacb5 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:05.436066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:05.441721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453c322-3e31-4d6e-b3ed-6e95c990ee9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:05.442366 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453c322-3e31-4d6e-b3ed-6e95c990ee9d None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:05.442650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453c322-3e31-4d6e-b3ed-6e95c990ee9d None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:05.442927 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453c322-3e31-4d6e-b3ed-6e95c990ee9d None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:05.443613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453c322-3e31-4d6e-b3ed-6e95c990ee9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:05.443918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453c322-3e31-4d6e-b3ed-6e95c990ee9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:05.445414 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453c322-3e31-4d6e-b3ed-6e95c990ee9d None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:05.457582 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 605/1204] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:50:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:05.463226 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0ed58e-6a89-4d20-8950-4fe2e87ce8cd None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:05.516020 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:05.561221 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0ed58e-6a89-4d20-8950-4fe2e87ce8cd None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:05.596059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5358556-23b3-4549-8186-37dde58ed483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:05.596664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5358556-23b3-4549-8186-37dde58ed483 None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:05.596946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5358556-23b3-4549-8186-37dde58ed483 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:05.597222 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5358556-23b3-4549-8186-37dde58ed483 None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:05.597909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5358556-23b3-4549-8186-37dde58ed483 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:05.599341 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5358556-23b3-4549-8186-37dde58ed483 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:05.800279 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 600/1205] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 2 08:50:05 2022] POST /identity/v3/users => generated 251 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:50:07.820422 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb5ea4-6e5c-4a97-9aff-d1984f17fa34 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:07.820977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb5ea4-6e5c-4a97-9aff-d1984f17fa34 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:07.820977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb5ea4-6e5c-4a97-9aff-d1984f17fa34 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:07.822869 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 606/1206] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:50:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:50:07.828130 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92174c60-accb-4901-94db-0e37b7be4955 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:07.828556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92174c60-accb-4901-94db-0e37b7be4955 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:07.828693 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92174c60-accb-4901-94db-0e37b7be4955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:07.847140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-92174c60-accb-4901-94db-0e37b7be4955 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:50:07.848346 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-92174c60-accb-4901-94db-0e37b7be4955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:50:07.914288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 601/1207] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:50:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:50:07.937537 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb5ea4-6e5c-4a97-9aff-d1984f17fa34 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:08.038290 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb5ea4-6e5c-4a97-9aff-d1984f17fa34 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:08.071286 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca5fbb-f658-4b76-9d91-8ac0ea1ae65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:08.071862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca5fbb-f658-4b76-9d91-8ac0ea1ae65a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:08.072636 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca5fbb-f658-4b76-9d91-8ac0ea1ae65a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:08.072914 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca5fbb-f658-4b76-9d91-8ac0ea1ae65a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:08.084439 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca5fbb-f658-4b76-9d91-8ac0ea1ae65a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:50:08.090046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-eaca5fbb-f658-4b76-9d91-8ac0ea1ae65a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:50:08.090046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:08.090046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:08.090046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:08.090046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:08.090046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:08.090046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:08.090046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:08.090046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:08.090046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:08.090046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:08.090046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:08.090046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:50:08.090046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:50:08.090046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:50:08.090046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:50:08.091190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:50:08.091190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:50:08.091190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:50:08.091190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:50:08.091190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:08.091190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:08.091190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:08.091190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:08.091190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:08.091190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:08.091190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:08.091190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:08.091190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:08.091190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:08.091190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:08.091190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:08.092376 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:08.092376 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:08.092376 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:08.092376 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:08.092376 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:50:08.092376 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:50:08.092376 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:50:08.092376 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:50:08.092376 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:50:08.092376 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:50:08.092376 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:50:08.092376 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:50:08.092376 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 607/1208] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 2 08:50:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:08.097230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92174c60-accb-4901-94db-0e37b7be4955 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:08.137082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:08.195422 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92174c60-accb-4901-94db-0e37b7be4955 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:08.228801 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae564a-c081-4bf7-8063-4fdd27be8b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:08.229425 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae564a-c081-4bf7-8063-4fdd27be8b0f None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:08.229959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae564a-c081-4bf7-8063-4fdd27be8b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:08.233032 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae564a-c081-4bf7-8063-4fdd27be8b0f None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:08.233032 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae564a-c081-4bf7-8063-4fdd27be8b0f None admin] RBAC: Adding query filter params (name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:08.233032 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae564a-c081-4bf7-8063-4fdd27be8b0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:08.233369 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae564a-c081-4bf7-8063-4fdd27be8b0f None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:08.241567 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 602/1209] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 2 08:50:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:08.247619 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca5fbb-f658-4b76-9d91-8ac0ea1ae65a None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:08.296777 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:08.346287 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca5fbb-f658-4b76-9d91-8ac0ea1ae65a None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:08.379840 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b0a21-46c6-43c0-801b-0cd408743d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:08.380464 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b0a21-46c6-43c0-801b-0cd408743d49 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:08.380742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b0a21-46c6-43c0-801b-0cd408743d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:08.381013 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815b0a21-46c6-43c0-801b-0cd408743d49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:08.392856 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815b0a21-46c6-43c0-801b-0cd408743d49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:08.394321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815b0a21-46c6-43c0-801b-0cd408743d49 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:08.408268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-815b0a21-46c6-43c0-801b-0cd408743d49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:08.408268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:08.408268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:08.408268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:08.408268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:08.408268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:08.408268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:08.408268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:08.408268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:08.408268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:08.408268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:08.408268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:08.408268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:50:08.408268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:50:08.408268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:50:08.414238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:50:08.414238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:08.414238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:08.414238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:08.414238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:08.414238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:08.414238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:08.414238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:08.414238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:08.414238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:08.414238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:08.414238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:08.414238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:08.414238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:08.414238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:08.414238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:08.414238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:08.415179 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:50:08.415179 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:50:08.415179 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:08.415179 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:50:08.415179 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 608/1210] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:50:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:08.415479 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae564a-c081-4bf7-8063-4fdd27be8b0f None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:08.516618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:08.519793 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae564a-c081-4bf7-8063-4fdd27be8b0f None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:08.553708 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7a905-46c7-4f7d-aefe-006cea655174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:08.554374 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7a905-46c7-4f7d-aefe-006cea655174 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:08.554764 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7a905-46c7-4f7d-aefe-006cea655174 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:08.555047 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed7a905-46c7-4f7d-aefe-006cea655174 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:08.555725 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7a905-46c7-4f7d-aefe-006cea655174 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:08.556031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7a905-46c7-4f7d-aefe-006cea655174 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:08.558221 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed7a905-46c7-4f7d-aefe-006cea655174 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:08.569129 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 603/1211] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:50:08.575074 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b0a21-46c6-43c0-801b-0cd408743d49 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:08.674462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815b0a21-46c6-43c0-801b-0cd408743d49 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:08.675959 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:08.709075 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcbbcf-9f96-4b0d-ab4f-7e1ed5e3edd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:08.709702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcbbcf-9f96-4b0d-ab4f-7e1ed5e3edd0 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:08.710012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcbbcf-9f96-4b0d-ab4f-7e1ed5e3edd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:08.710289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcbbcf-9f96-4b0d-ab4f-7e1ed5e3edd0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:08.717862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fcbbcf-9f96-4b0d-ab4f-7e1ed5e3edd0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:08.722771 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fcbbcf-9f96-4b0d-ab4f-7e1ed5e3edd0 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:08.730968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-92fcbbcf-9f96-4b0d-ab4f-7e1ed5e3edd0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 08:50:08.730968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:08.730968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:08.730968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:08.730968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:08.730968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:08.730968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:08.730968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:08.730968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:08.730968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:08.730968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:08.730968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:08.730968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:50:08.730968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:50:08.730968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:50:08.730968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:50:08.732116 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:08.732116 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:08.732116 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:50:08.732116 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:50:08.732116 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:50:08.732116 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:50:08.732116 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:08.732116 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:08.732116 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:08.732116 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:08.732116 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:08.732116 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:08.732116 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:08.732116 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:08.732116 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:08.732116 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:08.732116 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:08.732116 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:08.733327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:08.733327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:08.733327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:50:08.733327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:50:08.733327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:50:08.733327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:50:08.733327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:50:08.733327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:50:08.733327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 08:50:08.733327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:50:08.733327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 609/1212] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:50:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:08.737900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7a905-46c7-4f7d-aefe-006cea655174 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:08.836512 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed7a905-46c7-4f7d-aefe-006cea655174 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:08.869577 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8e17c-e7be-4189-a19d-71a5615dbd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:08.870211 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8e17c-e7be-4189-a19d-71a5615dbd30 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:08.870490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8e17c-e7be-4189-a19d-71a5615dbd30 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:08.870757 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8e17c-e7be-4189-a19d-71a5615dbd30 None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:08.871528 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8e17c-e7be-4189-a19d-71a5615dbd30 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:08.871830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8e17c-e7be-4189-a19d-71a5615dbd30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:08.873235 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8e17c-e7be-4189-a19d-71a5615dbd30 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:08.889732 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 604/1213] 10.222.0.46 () {56 vars in 1143 bytes} [Tue Aug 2 08:50:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:08.895669 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcbbcf-9f96-4b0d-ab4f-7e1ed5e3edd0 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:08.997950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcbbcf-9f96-4b0d-ab4f-7e1ed5e3edd0 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:09.032973 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db816b6-6ece-4a4a-a9fa-244149dd38c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:09.033610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db816b6-6ece-4a4a-a9fa-244149dd38c4 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:09.033921 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db816b6-6ece-4a4a-a9fa-244149dd38c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:09.034194 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db816b6-6ece-4a4a-a9fa-244149dd38c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:09.063338 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:09.099990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db816b6-6ece-4a4a-a9fa-244149dd38c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:09.101422 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db816b6-6ece-4a4a-a9fa-244149dd38c4 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:09.197950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 610/1214] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:50:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:50:09.204176 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8e17c-e7be-4189-a19d-71a5615dbd30 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:09.223437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:09.303039 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8e17c-e7be-4189-a19d-71a5615dbd30 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:09.337192 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91f1a73-907b-4cec-a5ed-2ad00fa271d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:09.338431 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f1a73-907b-4cec-a5ed-2ad00fa271d3 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:09.338713 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f1a73-907b-4cec-a5ed-2ad00fa271d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:09.338983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f1a73-907b-4cec-a5ed-2ad00fa271d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:09.348832 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91f1a73-907b-4cec-a5ed-2ad00fa271d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:09.350290 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91f1a73-907b-4cec-a5ed-2ad00fa271d3 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:09.364538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-a91f1a73-907b-4cec-a5ed-2ad00fa271d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:09.364538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:09.364538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:09.364538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:09.364538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:09.364538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:09.364538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:09.364538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:09.364538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:09.364538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:09.364538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:09.364538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:09.364538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:50:09.364538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:50:09.364538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:50:09.365715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:50:09.365715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:09.365715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:09.365715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:09.365715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:09.365715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:09.365715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:09.365715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:09.365715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:09.365715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:09.365715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:09.365715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:09.365715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:09.365715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:09.365715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:09.365715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:09.365715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:09.366931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:50:09.366931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:50:09.366931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:09.366931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:50:09.366931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 605/1215] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:50:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:09.373307 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db816b6-6ece-4a4a-a9fa-244149dd38c4 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:09.434576 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:09.472527 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db816b6-6ece-4a4a-a9fa-244149dd38c4 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:09.505529 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90acd5-06f6-4f66-93af-b3801a1cfd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:09.506175 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90acd5-06f6-4f66-93af-b3801a1cfd3b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:09.506459 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90acd5-06f6-4f66-93af-b3801a1cfd3b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:09.506741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d90acd5-06f6-4f66-93af-b3801a1cfd3b None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:09.507428 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d90acd5-06f6-4f66-93af-b3801a1cfd3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:09.507734 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d90acd5-06f6-4f66-93af-b3801a1cfd3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:09.509093 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d90acd5-06f6-4f66-93af-b3801a1cfd3b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:09.520778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 611/1216] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:50:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:09.527172 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91f1a73-907b-4cec-a5ed-2ad00fa271d3 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:09.596296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:09.625688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91f1a73-907b-4cec-a5ed-2ad00fa271d3 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:09.658514 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55927a0b-8b2f-4438-b526-656cce031baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:09.659146 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55927a0b-8b2f-4438-b526-656cce031baf None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:09.659424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55927a0b-8b2f-4438-b526-656cce031baf None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:09.659695 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55927a0b-8b2f-4438-b526-656cce031baf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:09.666190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55927a0b-8b2f-4438-b526-656cce031baf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:09.667618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55927a0b-8b2f-4438-b526-656cce031baf None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:09.678618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-55927a0b-8b2f-4438-b526-656cce031baf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:50:09.678618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:09.678618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:09.678618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:09.678618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:09.678618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:09.678618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:09.678618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:09.678618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:09.678618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:09.678618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:09.678618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:09.678618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:50:09.678618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:50:09.678618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:50:09.679738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:50:09.679738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:09.679738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:09.679738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:09.679738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:09.679738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:09.679738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:09.679738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:09.679738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:09.679738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:09.679738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:09.679738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:09.679738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:09.679738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:09.679738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:09.679738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:09.679738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:09.680831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:50:09.680831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:50:09.680831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:50:09.680831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:50:09.680831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:50:09.680831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:50:09.680831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:50:09.680831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:50:09.680831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 606/1217] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:50:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:09.686298 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90acd5-06f6-4f66-93af-b3801a1cfd3b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:09.786181 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d90acd5-06f6-4f66-93af-b3801a1cfd3b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:09.808968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:09.820619 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f8f20-7097-4a03-bd8f-9b1d678284bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:09.821232 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f8f20-7097-4a03-bd8f-9b1d678284bd None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:09.821512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f8f20-7097-4a03-bd8f-9b1d678284bd None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:09.821788 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f8f20-7097-4a03-bd8f-9b1d678284bd None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:09.823314 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f8f20-7097-4a03-bd8f-9b1d678284bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:09.823624 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f8f20-7097-4a03-bd8f-9b1d678284bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:09.825309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f8f20-7097-4a03-bd8f-9b1d678284bd None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:09.839358 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 612/1218] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:50:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:09.845977 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55927a0b-8b2f-4438-b526-656cce031baf None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:09.944558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55927a0b-8b2f-4438-b526-656cce031baf None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:09.972436 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:09.978044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c53e4-7411-4808-9434-f3b28f4d31b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:09.978650 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c53e4-7411-4808-9434-f3b28f4d31b5 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:09.978929 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c53e4-7411-4808-9434-f3b28f4d31b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:09.979200 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c53e4-7411-4808-9434-f3b28f4d31b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:09.979966 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c53e4-7411-4808-9434-f3b28f4d31b5 None admin] RBAC: Adding query filter params (role.id=391becda2bf04438b8d1763ab99acf1b, scope.project.id=4227a5b804c0414c99d407bb1415cf49, user.id=cad5cc9927374836b0cc74db85b686fd) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:09.980270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c53e4-7411-4808-9434-f3b28f4d31b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:09.981673 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c53e4-7411-4808-9434-f3b28f4d31b5 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:09.986982 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 607/1219] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:50:09 2022] GET /identity/v3/role_assignments?user.id=cad5cc9927374836b0cc74db85b686fd&scope.project.id=4227a5b804c0414c99d407bb1415cf49&role.id=391becda2bf04438b8d1763ab99acf1b => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:11.973420 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c1418-1119-4f64-8653-dc626f680086 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:11.973942 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c1418-1119-4f64-8653-dc626f680086 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:11.974062 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3c1418-1119-4f64-8653-dc626f680086 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:11.975367 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 613/1220] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:50:11.980625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccb22f-787c-405a-98f4-a4cc1944ae16 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:11.981021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccb22f-787c-405a-98f4-a4cc1944ae16 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:11.981182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccb22f-787c-405a-98f4-a4cc1944ae16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:11.999402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-06ccb22f-787c-405a-98f4-a4cc1944ae16 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:50:12.000675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-06ccb22f-787c-405a-98f4-a4cc1944ae16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:50:12.065215 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 608/1221] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:50:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:50:12.088516 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c1418-1119-4f64-8653-dc626f680086 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:12.188552 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3c1418-1119-4f64-8653-dc626f680086 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:12.223824 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3c69e-58a8-41d3-92d8-2742ced2a579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:12.224467 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3c69e-58a8-41d3-92d8-2742ced2a579 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:12.224747 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3c69e-58a8-41d3-92d8-2742ced2a579 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:12.225019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3c69e-58a8-41d3-92d8-2742ced2a579 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:12.235141 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3c69e-58a8-41d3-92d8-2742ced2a579 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:50:12.240002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-bce3c69e-58a8-41d3-92d8-2742ced2a579 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:50:12.240002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:12.240002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:12.240002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:12.240002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:12.240002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:12.240002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:12.240002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:12.240002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:12.240002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:12.240002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:12.240002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:12.240002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:50:12.240002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:50:12.240002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:50:12.240002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:50:12.241034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:50:12.241034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:50:12.241034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:50:12.241034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:50:12.241034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:12.241034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:12.241034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:12.241034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:12.241034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:12.241034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:12.241034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:12.241034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:12.241034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:12.241034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:12.241034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:12.241034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:12.241936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:12.241936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:12.241936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:12.241936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:12.241936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:50:12.241936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:50:12.241936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:50:12.241936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:50:12.241936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:50:12.241936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:50:12.241936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:50:12.241936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:50:12.241936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 614/1222] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 2 08:50:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:12.248060 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ccb22f-787c-405a-98f4-a4cc1944ae16 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:12.347918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ccb22f-787c-405a-98f4-a4cc1944ae16 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:12.373649 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:12.383861 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8d74e-89ba-4784-b9b1-190c2a8e9038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:12.384478 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8d74e-89ba-4784-b9b1-190c2a8e9038 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:12.384764 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8d74e-89ba-4784-b9b1-190c2a8e9038 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:12.385043 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd8d74e-89ba-4784-b9b1-190c2a8e9038 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:12.385756 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd8d74e-89ba-4784-b9b1-190c2a8e9038 None admin] RBAC: Adding query filter params (name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:12.386090 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd8d74e-89ba-4784-b9b1-190c2a8e9038 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:12.387458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd8d74e-89ba-4784-b9b1-190c2a8e9038 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:12.395331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 609/1223] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 2 08:50:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:12.401281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3c69e-58a8-41d3-92d8-2742ced2a579 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:12.445655 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:12.500132 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3c69e-58a8-41d3-92d8-2742ced2a579 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:12.534470 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77857f43-d1f1-4925-a445-a73d86c2f312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:12.535107 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77857f43-d1f1-4925-a445-a73d86c2f312 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:12.535389 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77857f43-d1f1-4925-a445-a73d86c2f312 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:12.535669 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77857f43-d1f1-4925-a445-a73d86c2f312 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:12.546292 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77857f43-d1f1-4925-a445-a73d86c2f312 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:12.547835 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77857f43-d1f1-4925-a445-a73d86c2f312 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:12.566188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-77857f43-d1f1-4925-a445-a73d86c2f312 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:12.566188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:12.566188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:12.566188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:12.566188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:12.566188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:12.566188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:12.566188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:12.566188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:12.566188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:12.566188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:12.566188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:12.566188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:50:12.566188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:50:12.566188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:50:12.567167 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:50:12.567167 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:12.567167 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:12.567167 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:12.567167 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:12.567167 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:12.567167 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:12.567167 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:12.567167 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:12.567167 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:12.567167 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:12.567167 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:12.567167 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:12.567167 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:12.567167 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:12.567167 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:12.567167 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:12.568119 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:50:12.568119 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:50:12.568119 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:12.568119 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:50:12.568119 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 615/1224] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:50:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:12.576987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8d74e-89ba-4784-b9b1-190c2a8e9038 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:12.679213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd8d74e-89ba-4784-b9b1-190c2a8e9038 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:12.713701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541347f-9b65-4f85-908a-5232e9059b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:12.714309 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541347f-9b65-4f85-908a-5232e9059b4e None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:12.714594 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541347f-9b65-4f85-908a-5232e9059b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:12.714989 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541347f-9b65-4f85-908a-5232e9059b4e None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:12.715677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541347f-9b65-4f85-908a-5232e9059b4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:12.715984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541347f-9b65-4f85-908a-5232e9059b4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:12.717408 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541347f-9b65-4f85-908a-5232e9059b4e None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:12.730832 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 610/1225] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:12.736622 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77857f43-d1f1-4925-a445-a73d86c2f312 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:12.829588 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:12.836037 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77857f43-d1f1-4925-a445-a73d86c2f312 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:12.869264 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484fb91-5900-432a-b385-23dfc3aa7624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:12.869898 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484fb91-5900-432a-b385-23dfc3aa7624 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:12.870185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484fb91-5900-432a-b385-23dfc3aa7624 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:12.870462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484fb91-5900-432a-b385-23dfc3aa7624 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:12.878270 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6484fb91-5900-432a-b385-23dfc3aa7624 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:12.879796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6484fb91-5900-432a-b385-23dfc3aa7624 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:12.891210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-6484fb91-5900-432a-b385-23dfc3aa7624 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 08:50:12.891210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:12.891210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:12.891210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:12.891210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:12.891210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:12.891210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:12.891210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:12.891210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:12.891210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:12.891210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:12.891210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:12.891210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:50:12.891210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:50:12.891210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:50:12.891210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:50:12.893535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:12.893535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:12.893535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:50:12.893535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:50:12.893535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:50:12.893535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:50:12.893535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:12.893535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:12.893535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:12.893535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:12.893535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:12.893535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:12.893535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:12.893535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:12.893535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:12.893535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:12.893535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:12.893535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:12.894796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:12.894796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:12.894796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:50:12.894796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:50:12.894796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:50:12.894796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:50:12.894796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:50:12.894796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:50:12.894796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 08:50:12.894796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:50:12.894796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 616/1226] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:50:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:12.898774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541347f-9b65-4f85-908a-5232e9059b4e None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:12.951023 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:13.003957 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541347f-9b65-4f85-908a-5232e9059b4e None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:13.037208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169bc45c-f2aa-4d4d-8ea4-28820bba25b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:13.037820 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bc45c-f2aa-4d4d-8ea4-28820bba25b1 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:13.038132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bc45c-f2aa-4d4d-8ea4-28820bba25b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:13.038410 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bc45c-f2aa-4d4d-8ea4-28820bba25b1 None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:13.039315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169bc45c-f2aa-4d4d-8ea4-28820bba25b1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:13.039975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169bc45c-f2aa-4d4d-8ea4-28820bba25b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:13.042915 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169bc45c-f2aa-4d4d-8ea4-28820bba25b1 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:13.060134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 611/1227] 10.222.0.46 () {56 vars in 1143 bytes} [Tue Aug 2 08:50:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:13.066179 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484fb91-5900-432a-b385-23dfc3aa7624 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:13.166401 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6484fb91-5900-432a-b385-23dfc3aa7624 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:13.200626 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15505c68-cdc1-49ba-bf8c-a5999e4fc574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:13.201261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15505c68-cdc1-49ba-bf8c-a5999e4fc574 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:13.201541 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15505c68-cdc1-49ba-bf8c-a5999e4fc574 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:13.201813 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15505c68-cdc1-49ba-bf8c-a5999e4fc574 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:13.212071 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15505c68-cdc1-49ba-bf8c-a5999e4fc574 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:13.214194 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15505c68-cdc1-49ba-bf8c-a5999e4fc574 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:13.228319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-15505c68-cdc1-49ba-bf8c-a5999e4fc574 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:13.228319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:13.228319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:13.228319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:13.228319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:13.228319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:13.228319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:13.228319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:13.228319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:13.228319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:13.228319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:13.228319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:13.228319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:50:13.228319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:50:13.228319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:50:13.229408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:50:13.229408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:13.229408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:13.229408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:13.229408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:13.229408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:13.229408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:13.229408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:13.229408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:13.229408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:13.229408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:13.229408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:13.229408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:13.229408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:13.229408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:13.229408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:13.229408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:13.230626 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:50:13.230626 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:50:13.230626 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:13.230626 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:50:13.230626 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 617/1228] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:50:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:13.235691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169bc45c-f2aa-4d4d-8ea4-28820bba25b1 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:13.340227 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169bc45c-f2aa-4d4d-8ea4-28820bba25b1 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:13.342788 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:13.373745 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec612f-39b3-4f02-8095-353991d697b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:13.374379 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec612f-39b3-4f02-8095-353991d697b6 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:13.374665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec612f-39b3-4f02-8095-353991d697b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:13.374940 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec612f-39b3-4f02-8095-353991d697b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:13.375616 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec612f-39b3-4f02-8095-353991d697b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:13.375923 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec612f-39b3-4f02-8095-353991d697b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:13.377316 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ec612f-39b3-4f02-8095-353991d697b6 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:13.389000 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 612/1229] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:13.395104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15505c68-cdc1-49ba-bf8c-a5999e4fc574 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:13.405072 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:13.493398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15505c68-cdc1-49ba-bf8c-a5999e4fc574 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:13.528275 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f6c91-a337-4d73-96a2-96759215d410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:13.529040 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f6c91-a337-4d73-96a2-96759215d410 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:13.529324 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f6c91-a337-4d73-96a2-96759215d410 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:13.529613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f6c91-a337-4d73-96a2-96759215d410 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:13.536198 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f6c91-a337-4d73-96a2-96759215d410 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:13.537790 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f6c91-a337-4d73-96a2-96759215d410 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:13.549317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-b64f6c91-a337-4d73-96a2-96759215d410 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:50:13.549317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:13.549317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:13.549317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:13.549317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:13.549317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:13.549317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:13.549317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:13.549317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:13.549317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:13.549317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:13.549317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:13.549317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:50:13.549317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:50:13.549317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:50:13.554850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:50:13.554850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:13.554850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:13.554850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:13.554850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:13.554850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:13.554850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:13.554850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:13.554850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:13.554850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:13.554850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:13.554850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:13.554850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:13.554850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:13.554850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:13.554850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:13.554850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:13.555910 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:50:13.555910 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:50:13.555910 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:50:13.555910 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:50:13.555910 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:50:13.555910 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:50:13.555910 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:50:13.555910 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:50:13.555910 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 618/1230] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:50:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:13.557035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec612f-39b3-4f02-8095-353991d697b6 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:13.655930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ec612f-39b3-4f02-8095-353991d697b6 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:13.689401 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc503cd-c0e9-42fa-a4ee-f595ab819e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:13.690044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc503cd-c0e9-42fa-a4ee-f595ab819e11 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:13.690329 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc503cd-c0e9-42fa-a4ee-f595ab819e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:13.690604 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc503cd-c0e9-42fa-a4ee-f595ab819e11 None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:13.691447 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc503cd-c0e9-42fa-a4ee-f595ab819e11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:13.691769 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc503cd-c0e9-42fa-a4ee-f595ab819e11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:13.693218 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc503cd-c0e9-42fa-a4ee-f595ab819e11 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:13.705803 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 613/1231] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:50:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:13.712098 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f6c91-a337-4d73-96a2-96759215d410 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:13.777996 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:13.812490 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f6c91-a337-4d73-96a2-96759215d410 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:13.844960 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a603bf-0660-403e-ade6-65cddc12016b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:13.845628 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a603bf-0660-403e-ade6-65cddc12016b None admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:13.845948 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a603bf-0660-403e-ade6-65cddc12016b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:13.846213 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a603bf-0660-403e-ade6-65cddc12016b None admin] PATH_INFO: `/v3/projects/4227a5b804c0414c99d407bb1415cf49/users/cad5cc9927374836b0cc74db85b686fd/roles/391becda2bf04438b8d1763ab99acf1b` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:13.860741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a603bf-0660-403e-ade6-65cddc12016b None admin] RBAC: Authorizing `identity:create_grant(project_id=4227a5b804c0414c99d407bb1415cf49, user_id=cad5cc9927374836b0cc74db85b686fd, role_id=391becda2bf04438b8d1763ab99acf1b)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:13.862221 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a603bf-0660-403e-ade6-65cddc12016b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:13.973292 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 619/1232] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:50:13 2022] PUT /identity/v3/projects/4227a5b804c0414c99d407bb1415cf49/users/cad5cc9927374836b0cc74db85b686fd/roles/391becda2bf04438b8d1763ab99acf1b => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:50:15.978187 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32a0ea-99bd-46af-bd7a-9a2bb1ffebc7 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:15.978711 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32a0ea-99bd-46af-bd7a-9a2bb1ffebc7 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:15.978892 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32a0ea-99bd-46af-bd7a-9a2bb1ffebc7 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:15.980155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 614/1233] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:50:15.985405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87521309-986d-4fd1-af47-5448c6f13dd0 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:15.985912 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87521309-986d-4fd1-af47-5448c6f13dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:15.986043 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87521309-986d-4fd1-af47-5448c6f13dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:16.004658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-87521309-986d-4fd1-af47-5448c6f13dd0 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:50:16.005855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-87521309-986d-4fd1-af47-5448c6f13dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:50:16.071370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 620/1234] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:50:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:50:16.094245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32a0ea-99bd-46af-bd7a-9a2bb1ffebc7 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:16.120107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:16.194478 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32a0ea-99bd-46af-bd7a-9a2bb1ffebc7 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:16.228875 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11a621-8391-4522-8fff-cbd9ccdec1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:16.229522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11a621-8391-4522-8fff-cbd9ccdec1b6 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:16.229801 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11a621-8391-4522-8fff-cbd9ccdec1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:16.230099 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11a621-8391-4522-8fff-cbd9ccdec1b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:16.239861 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11a621-8391-4522-8fff-cbd9ccdec1b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:50:16.245302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-de11a621-8391-4522-8fff-cbd9ccdec1b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:50:16.245302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:16.245302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:16.245302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:16.245302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:16.245302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:16.245302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:16.245302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:16.245302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:16.245302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:16.245302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:16.245302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:16.245302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:50:16.245302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:50:16.245302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:50:16.245302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:50:16.252983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:50:16.252983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:50:16.252983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:50:16.252983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:50:16.252983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:16.252983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:16.252983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:16.252983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:16.252983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:16.252983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:16.252983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:16.252983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:16.252983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:16.252983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:16.252983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:16.252983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:16.253978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:16.253978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:16.253978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:16.253978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:16.253978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:50:16.253978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:50:16.253978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:50:16.253978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:50:16.253978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:50:16.253978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:50:16.253978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 08:50:16.253978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:50:16.253978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 615/1235] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 2 08:50:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:16.254844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87521309-986d-4fd1-af47-5448c6f13dd0 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:16.302968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:16.356421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87521309-986d-4fd1-af47-5448c6f13dd0 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:16.390090 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8450c-6392-456d-bc14-3539185bbd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:16.390718 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8450c-6392-456d-bc14-3539185bbd79 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:16.391002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8450c-6392-456d-bc14-3539185bbd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:16.391280 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8450c-6392-456d-bc14-3539185bbd79 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:16.391993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8450c-6392-456d-bc14-3539185bbd79 None admin] RBAC: Adding query filter params (name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:16.392301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8450c-6392-456d-bc14-3539185bbd79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:16.393763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8450c-6392-456d-bc14-3539185bbd79 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:16.401603 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 621/1236] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 2 08:50:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:50:16.407384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11a621-8391-4522-8fff-cbd9ccdec1b6 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:16.494853 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:16.510422 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11a621-8391-4522-8fff-cbd9ccdec1b6 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:16.544979 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cadc02c-b164-44cc-9ad7-d01885e861e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:16.545608 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cadc02c-b164-44cc-9ad7-d01885e861e9 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:16.545920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cadc02c-b164-44cc-9ad7-d01885e861e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:16.546188 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cadc02c-b164-44cc-9ad7-d01885e861e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:16.555855 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cadc02c-b164-44cc-9ad7-d01885e861e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:16.557644 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cadc02c-b164-44cc-9ad7-d01885e861e9 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:16.571789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-4cadc02c-b164-44cc-9ad7-d01885e861e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:16.571789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:16.571789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:16.571789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:16.571789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:16.571789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:16.571789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:16.571789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:16.571789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:16.571789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:16.571789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:16.571789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:16.571789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:50:16.571789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:50:16.571789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:50:16.572864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:50:16.572864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:16.572864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:16.572864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:16.572864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:16.572864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:16.572864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:16.572864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:16.572864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:16.572864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:16.572864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:16.572864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:16.572864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:16.572864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:16.572864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:16.572864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:16.572864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:16.573799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:50:16.573799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:50:16.573799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:16.573799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:50:16.573799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 616/1237] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:50:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:16.579166 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8450c-6392-456d-bc14-3539185bbd79 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:16.679628 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8450c-6392-456d-bc14-3539185bbd79 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:16.688358 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:16.712323 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1993a-1e32-4729-9a6a-8f0ffa5fe99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:16.713607 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1993a-1e32-4729-9a6a-8f0ffa5fe99a None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:16.713898 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1993a-1e32-4729-9a6a-8f0ffa5fe99a None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:16.714255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b1993a-1e32-4729-9a6a-8f0ffa5fe99a None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:16.714934 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b1993a-1e32-4729-9a6a-8f0ffa5fe99a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:16.715318 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b1993a-1e32-4729-9a6a-8f0ffa5fe99a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:16.716666 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b1993a-1e32-4729-9a6a-8f0ffa5fe99a None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:16.727895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 622/1238] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:16.734187 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cadc02c-b164-44cc-9ad7-d01885e861e9 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:16.835104 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cadc02c-b164-44cc-9ad7-d01885e861e9 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:16.867947 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e622bf-68b7-4474-ad70-28e8758c7840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:16.868581 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e622bf-68b7-4474-ad70-28e8758c7840 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:16.868896 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e622bf-68b7-4474-ad70-28e8758c7840 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:16.869170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e622bf-68b7-4474-ad70-28e8758c7840 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:16.871806 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:16.876568 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e622bf-68b7-4474-ad70-28e8758c7840 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:16.878035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e622bf-68b7-4474-ad70-28e8758c7840 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:16.889497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-d5e622bf-68b7-4474-ad70-28e8758c7840 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 08:50:16.889497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:16.889497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:16.889497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:16.889497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:16.889497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:16.889497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:16.889497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:16.889497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:16.889497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:16.889497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:16.889497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:16.889497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:50:16.889497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:50:16.889497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:50:16.889497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:50:16.890657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:16.890657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:16.890657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:50:16.890657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:50:16.890657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:50:16.890657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:50:16.890657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:16.890657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:16.890657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:16.890657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:16.890657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:16.890657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:16.890657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:16.890657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:16.890657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:16.890657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:16.890657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:16.890657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:16.891696 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:16.891696 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:16.891696 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:50:16.891696 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:50:16.891696 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:50:16.891696 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:50:16.891696 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:50:16.891696 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:50:16.891696 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 08:50:16.891696 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:50:16.891696 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 617/1239] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:50:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:16.896644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1993a-1e32-4729-9a6a-8f0ffa5fe99a None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:16.997154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b1993a-1e32-4729-9a6a-8f0ffa5fe99a None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:17.031169 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da2bef-761d-4c5c-b322-03ca54319ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:17.031783 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da2bef-761d-4c5c-b322-03ca54319ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:17.032108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da2bef-761d-4c5c-b322-03ca54319ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:17.032384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da2bef-761d-4c5c-b322-03ca54319ce4 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:17.033166 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da2bef-761d-4c5c-b322-03ca54319ce4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:17.033471 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da2bef-761d-4c5c-b322-03ca54319ce4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:17.035096 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7da2bef-761d-4c5c-b322-03ca54319ce4 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:17.051741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 623/1240] 10.222.0.46 () {56 vars in 1143 bytes} [Tue Aug 2 08:50:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:17.057629 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e622bf-68b7-4474-ad70-28e8758c7840 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:17.157421 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e622bf-68b7-4474-ad70-28e8758c7840 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:17.190209 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4a7ca-6e2b-41f5-9fce-030c48c8f847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:17.190819 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4a7ca-6e2b-41f5-9fce-030c48c8f847 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:17.191104 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4a7ca-6e2b-41f5-9fce-030c48c8f847 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:17.191381 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4a7ca-6e2b-41f5-9fce-030c48c8f847 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:17.250894 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:17.257255 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4a7ca-6e2b-41f5-9fce-030c48c8f847 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:17.258738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e4a7ca-6e2b-41f5-9fce-030c48c8f847 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:17.355245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 618/1241] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:50:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:50:17.361202 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da2bef-761d-4c5c-b322-03ca54319ce4 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:17.389954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:17.463260 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7da2bef-761d-4c5c-b322-03ca54319ce4 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:17.496460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d95012-7931-4930-995c-cc892a8c6ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:17.497089 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d95012-7931-4930-995c-cc892a8c6ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:17.497368 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d95012-7931-4930-995c-cc892a8c6ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:17.497696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d95012-7931-4930-995c-cc892a8c6ed3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:17.507714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d95012-7931-4930-995c-cc892a8c6ed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:17.509287 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d95012-7931-4930-995c-cc892a8c6ed3 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:17.524532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-47d95012-7931-4930-995c-cc892a8c6ed3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:17.524532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:17.524532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:17.524532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:17.524532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:17.524532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:17.524532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:17.524532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:17.524532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:17.524532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:17.524532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:17.524532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:17.524532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:50:17.524532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:50:17.524532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:50:17.533342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:50:17.533342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:17.533342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:17.533342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:17.533342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:17.533342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:17.533342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:17.533342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:17.533342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:17.533342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:17.533342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:17.533342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:17.533342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:17.533342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:17.533342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:17.533342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:17.533342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:17.534488 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:50:17.534488 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:50:17.534488 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:17.534488 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:50:17.534488 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 624/1242] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:50:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:17.534790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4a7ca-6e2b-41f5-9fce-030c48c8f847 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:17.624115 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:17.629630 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4a7ca-6e2b-41f5-9fce-030c48c8f847 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:17.661524 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0010b7-d267-43ac-89e4-350c18387117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:17.662159 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0010b7-d267-43ac-89e4-350c18387117 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:17.662491 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0010b7-d267-43ac-89e4-350c18387117 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:17.662772 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0010b7-d267-43ac-89e4-350c18387117 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:17.663455 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0010b7-d267-43ac-89e4-350c18387117 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:17.663757 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0010b7-d267-43ac-89e4-350c18387117 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:17.665116 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0010b7-d267-43ac-89e4-350c18387117 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:17.676511 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 619/1243] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:50:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:17.683008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d95012-7931-4930-995c-cc892a8c6ed3 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:17.764395 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:17.785144 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d95012-7931-4930-995c-cc892a8c6ed3 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:17.818580 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40833f-d481-4378-93b2-40d56389d8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:17.819210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40833f-d481-4378-93b2-40d56389d8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:17.819487 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40833f-d481-4378-93b2-40d56389d8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:17.819759 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa40833f-d481-4378-93b2-40d56389d8b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:17.826927 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa40833f-d481-4378-93b2-40d56389d8b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:17.833353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa40833f-d481-4378-93b2-40d56389d8b7 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:17.839720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-fa40833f-d481-4378-93b2-40d56389d8b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:50:17.839720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:17.839720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:17.839720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:17.839720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:17.839720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:17.839720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:17.839720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:17.839720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:17.839720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:17.839720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:17.839720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:17.839720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:50:17.839720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:50:17.839720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:50:17.840768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:50:17.840768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:17.840768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:17.840768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:17.840768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:17.840768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:17.840768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:17.840768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:17.840768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:17.840768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:17.840768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:17.840768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:17.840768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:17.840768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:17.840768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:17.840768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:17.840768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:17.841791 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:50:17.841791 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:50:17.841791 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:50:17.841791 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:50:17.841791 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:50:17.841791 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:50:17.841791 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:50:17.841791 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:50:17.841791 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 625/1244] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:50:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:17.846822 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0010b7-d267-43ac-89e4-350c18387117 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:17.946888 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0010b7-d267-43ac-89e4-350c18387117 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:17.980848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6044fc-a2d9-4949-acc7-d9efa44de06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:17.981460 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6044fc-a2d9-4949-acc7-d9efa44de06d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:17.981742 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6044fc-a2d9-4949-acc7-d9efa44de06d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:17.982054 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6044fc-a2d9-4949-acc7-d9efa44de06d None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:17.982797 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6044fc-a2d9-4949-acc7-d9efa44de06d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:17.983107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6044fc-a2d9-4949-acc7-d9efa44de06d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:17.984562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6044fc-a2d9-4949-acc7-d9efa44de06d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:17.996276 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 620/1245] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:50:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:18.002516 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40833f-d481-4378-93b2-40d56389d8b7 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:18.099654 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa40833f-d481-4378-93b2-40d56389d8b7 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:18.132454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee94320-afed-48b3-952d-fed20a79082c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:18.133061 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee94320-afed-48b3-952d-fed20a79082c None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:18.133339 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee94320-afed-48b3-952d-fed20a79082c None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:18.133608 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee94320-afed-48b3-952d-fed20a79082c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:18.134438 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee94320-afed-48b3-952d-fed20a79082c None admin] RBAC: Adding query filter params (role.id=391becda2bf04438b8d1763ab99acf1b, scope.project.id=4227a5b804c0414c99d407bb1415cf49, user.id=cad5cc9927374836b0cc74db85b686fd) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:18.134741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee94320-afed-48b3-952d-fed20a79082c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:18.136155 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee94320-afed-48b3-952d-fed20a79082c None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:18.141692 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 626/1246] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:50:18 2022] GET /identity/v3/role_assignments?user.id=cad5cc9927374836b0cc74db85b686fd&scope.project.id=4227a5b804c0414c99d407bb1415cf49&role.id=391becda2bf04438b8d1763ab99acf1b => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:20.179901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0169b45-6629-4d4f-a595-d323f5a55729 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:20.180481 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0169b45-6629-4d4f-a595-d323f5a55729 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:20.180481 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0169b45-6629-4d4f-a595-d323f5a55729 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:20.182374 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 621/1247] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:50:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:50:20.187401 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d54a40-62ed-4638-88d3-8e2abc5b7eeb None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:20.187820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d54a40-62ed-4638-88d3-8e2abc5b7eeb None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:20.187964 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d54a40-62ed-4638-88d3-8e2abc5b7eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:20.194493 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:20.207495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-c1d54a40-62ed-4638-88d3-8e2abc5b7eeb None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:50:20.208694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c1d54a40-62ed-4638-88d3-8e2abc5b7eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:50:20.275638 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 627/1248] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:50:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:50:20.298797 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0169b45-6629-4d4f-a595-d323f5a55729 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:20.317067 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:20.399569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0169b45-6629-4d4f-a595-d323f5a55729 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:20.432406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad6a5da-144e-4d30-b3b1-e0f28e9804d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:20.433038 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6a5da-144e-4d30-b3b1-e0f28e9804d7 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:20.433321 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6a5da-144e-4d30-b3b1-e0f28e9804d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:20.433599 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6a5da-144e-4d30-b3b1-e0f28e9804d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:20.443504 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad6a5da-144e-4d30-b3b1-e0f28e9804d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:50:20.449108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-dad6a5da-144e-4d30-b3b1-e0f28e9804d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:50:20.449108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:20.449108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:20.449108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:20.449108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:20.449108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:20.449108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:20.449108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:20.449108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:20.449108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:20.449108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:20.449108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:20.449108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:50:20.449108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:50:20.449108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:50:20.449108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:50:20.450585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:50:20.450585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:50:20.450585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:50:20.450585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:50:20.450585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:20.450585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:20.450585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:20.450585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:20.450585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:20.450585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:20.450585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:20.450585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:20.450585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:20.450585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:20.450585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:20.450585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:20.451855 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:20.451855 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:20.451855 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:20.451855 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:20.451855 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:50:20.451855 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:50:20.451855 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:50:20.451855 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:50:20.451855 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:50:20.451855 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:50:20.451855 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:50:20.451855 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:50:20.451855 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 622/1249] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:50:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:20.456644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d54a40-62ed-4638-88d3-8e2abc5b7eeb None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:20.555270 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d54a40-62ed-4638-88d3-8e2abc5b7eeb None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:20.584431 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:20.589337 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9800d59-affc-4062-9b40-cd4029c7cc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:20.589981 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9800d59-affc-4062-9b40-cd4029c7cc04 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:20.590266 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9800d59-affc-4062-9b40-cd4029c7cc04 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:20.590541 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9800d59-affc-4062-9b40-cd4029c7cc04 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:20.591246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9800d59-affc-4062-9b40-cd4029c7cc04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:20.591549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9800d59-affc-4062-9b40-cd4029c7cc04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:20.592908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9800d59-affc-4062-9b40-cd4029c7cc04 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:20.600725 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 628/1250] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:50:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:20.606971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad6a5da-144e-4d30-b3b1-e0f28e9804d7 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:20.682601 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:20.704667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad6a5da-144e-4d30-b3b1-e0f28e9804d7 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:20.738169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1784f5-97b0-478a-be1e-424bd0e80a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:20.738805 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1784f5-97b0-478a-be1e-424bd0e80a40 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:20.739087 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1784f5-97b0-478a-be1e-424bd0e80a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:20.739365 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1784f5-97b0-478a-be1e-424bd0e80a40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:20.750092 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1784f5-97b0-478a-be1e-424bd0e80a40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:20.751554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1784f5-97b0-478a-be1e-424bd0e80a40 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:20.765909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-9e1784f5-97b0-478a-be1e-424bd0e80a40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:20.765909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:20.765909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:20.765909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:20.765909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:20.765909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:20.765909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:20.765909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:20.765909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:20.765909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:20.765909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:20.765909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:20.765909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:50:20.765909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:50:20.765909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:50:20.766991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:50:20.766991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:20.766991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:20.766991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:20.766991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:20.766991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:20.766991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:20.766991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:20.766991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:20.766991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:20.766991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:20.766991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:20.766991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:20.766991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:20.766991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:20.766991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:20.766991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:20.768262 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:50:20.768262 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:50:20.768262 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:20.768262 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:50:20.768262 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 623/1251] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:50:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:20.772829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9800d59-affc-4062-9b40-cd4029c7cc04 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:20.872737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9800d59-affc-4062-9b40-cd4029c7cc04 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:20.904571 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4e580-c464-4a35-8fe0-9a01e7e8b46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:20.905209 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4e580-c464-4a35-8fe0-9a01e7e8b46b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:20.905489 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4e580-c464-4a35-8fe0-9a01e7e8b46b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:20.905764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4e580-c464-4a35-8fe0-9a01e7e8b46b None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:20.906476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f4e580-c464-4a35-8fe0-9a01e7e8b46b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:20.911722 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f4e580-c464-4a35-8fe0-9a01e7e8b46b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:20.911722 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f4e580-c464-4a35-8fe0-9a01e7e8b46b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:20.919966 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 629/1252] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:20.925768 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1784f5-97b0-478a-be1e-424bd0e80a40 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:21.028597 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1784f5-97b0-478a-be1e-424bd0e80a40 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:21.054084 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:21.061621 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc854bc5-a8c7-498d-be9f-4bc28a7ecd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:21.062287 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc854bc5-a8c7-498d-be9f-4bc28a7ecd42 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:21.062569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc854bc5-a8c7-498d-be9f-4bc28a7ecd42 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:21.062842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc854bc5-a8c7-498d-be9f-4bc28a7ecd42 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:21.070035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc854bc5-a8c7-498d-be9f-4bc28a7ecd42 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:21.076608 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc854bc5-a8c7-498d-be9f-4bc28a7ecd42 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:21.082607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-dc854bc5-a8c7-498d-be9f-4bc28a7ecd42 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 08:50:21.082607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:21.082607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:21.082607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:21.082607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:21.082607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:21.082607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:21.082607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:21.082607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:21.082607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:21.082607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:21.082607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:21.082607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:50:21.082607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:50:21.082607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:50:21.082607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:50:21.083783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:21.083783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:21.083783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:50:21.083783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:50:21.083783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:50:21.083783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:50:21.083783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:21.083783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:21.083783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:21.083783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:21.083783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:21.083783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:21.083783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:21.083783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:21.083783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:21.083783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:21.083783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:21.083783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:21.084784 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:21.084784 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:21.084784 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:50:21.084784 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:50:21.084784 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:50:21.084784 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:50:21.084784 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:50:21.084784 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:50:21.084784 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 08:50:21.084784 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:50:21.084784 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 624/1253] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:50:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:21.090145 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4e580-c464-4a35-8fe0-9a01e7e8b46b None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:21.146998 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:21.189778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4e580-c464-4a35-8fe0-9a01e7e8b46b None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:21.224108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3c383-c11b-449a-ba88-decee515f747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:21.224719 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3c383-c11b-449a-ba88-decee515f747 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:21.224996 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3c383-c11b-449a-ba88-decee515f747 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:21.225267 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba3c383-c11b-449a-ba88-decee515f747 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:21.226074 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba3c383-c11b-449a-ba88-decee515f747 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:21.226378 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba3c383-c11b-449a-ba88-decee515f747 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:21.227797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba3c383-c11b-449a-ba88-decee515f747 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:21.244795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 630/1254] 10.222.0.46 () {56 vars in 1143 bytes} [Tue Aug 2 08:50:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:21.250655 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc854bc5-a8c7-498d-be9f-4bc28a7ecd42 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:21.350475 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc854bc5-a8c7-498d-be9f-4bc28a7ecd42 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:21.382601 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d0b6d-98ff-4458-9ae4-fcb100bcc8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:21.383205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d0b6d-98ff-4458-9ae4-fcb100bcc8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:21.383595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d0b6d-98ff-4458-9ae4-fcb100bcc8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:21.383880 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d0b6d-98ff-4458-9ae4-fcb100bcc8a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:21.439032 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:21.451796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d0b6d-98ff-4458-9ae4-fcb100bcc8a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:21.453257 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d0b6d-98ff-4458-9ae4-fcb100bcc8a6 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:21.551219 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 625/1255] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:50:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:50:21.557185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3c383-c11b-449a-ba88-decee515f747 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:21.656282 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba3c383-c11b-449a-ba88-decee515f747 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:21.668990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:21.689478 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee2227-7905-432a-9280-c109eda98a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:21.690272 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee2227-7905-432a-9280-c109eda98a73 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:21.690560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee2227-7905-432a-9280-c109eda98a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:21.690841 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee2227-7905-432a-9280-c109eda98a73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:21.701338 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee2227-7905-432a-9280-c109eda98a73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:21.702803 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee2227-7905-432a-9280-c109eda98a73 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:21.717296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-d4ee2227-7905-432a-9280-c109eda98a73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:21.717296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:21.717296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:21.717296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:21.717296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:21.717296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:21.717296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:21.717296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:21.717296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:21.717296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:21.717296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:21.717296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:21.717296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:50:21.717296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:50:21.717296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:50:21.718425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:50:21.718425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:21.718425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:21.718425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:21.718425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:21.718425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:21.718425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:21.718425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:21.718425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:21.718425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:21.718425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:21.718425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:21.718425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:21.718425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:21.718425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:21.718425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:21.718425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:21.719565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:50:21.719565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:50:21.719565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:21.719565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:50:21.719565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 631/1256] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:50:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:21.724254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d0b6d-98ff-4458-9ae4-fcb100bcc8a6 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:21.809947 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:21.823116 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d0b6d-98ff-4458-9ae4-fcb100bcc8a6 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:21.856081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f46f9-c5af-4b8e-a1cb-08e088f907a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:21.856692 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f46f9-c5af-4b8e-a1cb-08e088f907a1 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:21.856977 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f46f9-c5af-4b8e-a1cb-08e088f907a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:21.857254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0f46f9-c5af-4b8e-a1cb-08e088f907a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:21.857966 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f46f9-c5af-4b8e-a1cb-08e088f907a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:21.858274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f46f9-c5af-4b8e-a1cb-08e088f907a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:21.859629 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0f46f9-c5af-4b8e-a1cb-08e088f907a1 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:21.871247 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 626/1257] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:50:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:21.877023 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee2227-7905-432a-9280-c109eda98a73 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:21.976457 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee2227-7905-432a-9280-c109eda98a73 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:22.009504 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c02c5f-5925-4301-8b00-4546d9f40057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:22.010177 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c02c5f-5925-4301-8b00-4546d9f40057 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:22.010982 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c02c5f-5925-4301-8b00-4546d9f40057 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:22.011263 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c02c5f-5925-4301-8b00-4546d9f40057 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:22.017739 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c02c5f-5925-4301-8b00-4546d9f40057 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:22.019208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c02c5f-5925-4301-8b00-4546d9f40057 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:22.030565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-96c02c5f-5925-4301-8b00-4546d9f40057 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:50:22.030565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:22.030565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:22.030565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:22.030565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:22.030565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:22.030565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:22.030565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:22.030565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:22.030565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:22.030565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:22.030565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:22.030565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:50:22.030565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:50:22.030565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:50:22.031645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:50:22.031645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:22.031645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:22.031645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:22.031645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:22.031645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:22.031645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:22.031645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:22.031645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:22.031645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:22.031645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:22.031645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:22.031645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:22.031645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:22.031645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:22.031645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:22.031645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:22.032940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:50:22.032940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:50:22.032940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:50:22.032940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:50:22.032940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:50:22.032940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:50:22.032940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:50:22.032940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:50:22.032940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 632/1258] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:50:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:22.037570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f46f9-c5af-4b8e-a1cb-08e088f907a1 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:22.135207 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0f46f9-c5af-4b8e-a1cb-08e088f907a1 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:22.167880 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee5f55-e0cc-4496-917e-8558dfd3bf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:22.168489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee5f55-e0cc-4496-917e-8558dfd3bf0b None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:22.168772 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee5f55-e0cc-4496-917e-8558dfd3bf0b None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:22.169046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee5f55-e0cc-4496-917e-8558dfd3bf0b None admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:22.169784 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee5f55-e0cc-4496-917e-8558dfd3bf0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:22.170135 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee5f55-e0cc-4496-917e-8558dfd3bf0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:22.171547 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee5f55-e0cc-4496-917e-8558dfd3bf0b None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:22.183612 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 627/1259] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:50:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:22.189916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c02c5f-5925-4301-8b00-4546d9f40057 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:22.215208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:22.291546 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c02c5f-5925-4301-8b00-4546d9f40057 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:22.326334 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064de5d-4c6d-4321-85c4-92756a385fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:22.326954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064de5d-4c6d-4321-85c4-92756a385fbc None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:22.327235 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064de5d-4c6d-4321-85c4-92756a385fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:22.327512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064de5d-4c6d-4321-85c4-92756a385fbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:22.328302 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064de5d-4c6d-4321-85c4-92756a385fbc None admin] RBAC: Adding query filter params (role.id=8ce0850e15804f3d811e88cc9841e673, scope.project.id=4227a5b804c0414c99d407bb1415cf49, user.id=cad5cc9927374836b0cc74db85b686fd) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:22.328613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064de5d-4c6d-4321-85c4-92756a385fbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:22.330071 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064de5d-4c6d-4321-85c4-92756a385fbc None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:22.335552 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 633/1260] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:50:22 2022] GET /identity/v3/role_assignments?user.id=cad5cc9927374836b0cc74db85b686fd&scope.project.id=4227a5b804c0414c99d407bb1415cf49&role.id=8ce0850e15804f3d811e88cc9841e673 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:24.343709 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a1329-1dc2-42b6-befa-00c08e33e552 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:24.344206 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a1329-1dc2-42b6-befa-00c08e33e552 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:24.344206 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a1329-1dc2-42b6-befa-00c08e33e552 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:24.345553 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 628/1261] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:50:24.350948 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86496e33-4f58-4cd6-ad8a-78b3fe40bbde None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:24.351398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86496e33-4f58-4cd6-ad8a-78b3fe40bbde None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:24.351489 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86496e33-4f58-4cd6-ad8a-78b3fe40bbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:24.370260 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-86496e33-4f58-4cd6-ad8a-78b3fe40bbde None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:50:24.371443 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-86496e33-4f58-4cd6-ad8a-78b3fe40bbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:50:24.437146 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 634/1262] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:50:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:50:24.459488 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a1329-1dc2-42b6-befa-00c08e33e552 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:24.470351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:24.560001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a1329-1dc2-42b6-befa-00c08e33e552 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:24.594384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5542bb-03e3-4817-85fe-019933b42e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:24.594954 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5542bb-03e3-4817-85fe-019933b42e05 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:24.595240 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5542bb-03e3-4817-85fe-019933b42e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:24.595520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5542bb-03e3-4817-85fe-019933b42e05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:24.605980 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5542bb-03e3-4817-85fe-019933b42e05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:50:24.610840 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-0a5542bb-03e3-4817-85fe-019933b42e05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:50:24.610840 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:24.610840 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:24.610840 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:24.610840 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:24.610840 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:24.610840 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:24.610840 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:24.610840 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:24.610840 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:24.610840 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:24.610840 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:24.610840 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:50:24.610840 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:50:24.610840 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:50:24.610840 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:50:24.612046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:50:24.612046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:50:24.612046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:50:24.612046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:50:24.612046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:24.612046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:24.612046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:24.612046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:24.612046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:24.612046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:24.612046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:24.612046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:24.612046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:24.612046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:24.612046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:24.612046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:24.619946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:24.619946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:24.619946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:24.619946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:24.619946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:50:24.619946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:50:24.619946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:50:24.619946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:50:24.619946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:50:24.619946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:50:24.619946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:50:24.619946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:50:24.619946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 629/1263] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:50:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:24.620826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86496e33-4f58-4cd6-ad8a-78b3fe40bbde None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:24.636841 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:24.716156 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86496e33-4f58-4cd6-ad8a-78b3fe40bbde None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:24.750932 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852cbe7-c7c0-4db0-8357-72b871ba96b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:24.751557 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852cbe7-c7c0-4db0-8357-72b871ba96b1 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:24.751842 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852cbe7-c7c0-4db0-8357-72b871ba96b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:24.752121 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3852cbe7-c7c0-4db0-8357-72b871ba96b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:24.752844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852cbe7-c7c0-4db0-8357-72b871ba96b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:24.753155 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852cbe7-c7c0-4db0-8357-72b871ba96b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:24.754566 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3852cbe7-c7c0-4db0-8357-72b871ba96b1 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:24.762817 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 635/1264] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:50:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:24.768767 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5542bb-03e3-4817-85fe-019933b42e05 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:24.837755 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:24.870443 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5542bb-03e3-4817-85fe-019933b42e05 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:24.902835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b7fcf-77b3-4433-899e-142c8934d8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:24.903477 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b7fcf-77b3-4433-899e-142c8934d8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:24.903795 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b7fcf-77b3-4433-899e-142c8934d8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:24.904069 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b7fcf-77b3-4433-899e-142c8934d8c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:24.914218 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b7fcf-77b3-4433-899e-142c8934d8c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:24.916229 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b7fcf-77b3-4433-899e-142c8934d8c6 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:24.930337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-e90b7fcf-77b3-4433-899e-142c8934d8c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:24.930337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:24.930337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:24.930337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:24.930337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:24.930337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:24.930337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:24.930337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:24.930337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:24.930337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:24.930337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:24.930337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:24.930337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:50:24.930337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:50:24.930337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:50:24.931437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:50:24.931437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:24.931437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:24.931437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:24.931437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:24.931437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:24.931437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:24.931437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:24.931437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:24.931437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:24.931437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:24.931437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:24.931437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:24.931437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:24.931437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:24.931437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:24.931437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:24.932597 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:50:24.932597 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:50:24.932597 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:24.932597 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:50:24.932597 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 630/1265] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:24.937305 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852cbe7-c7c0-4db0-8357-72b871ba96b1 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:25.016146 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:25.034714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3852cbe7-c7c0-4db0-8357-72b871ba96b1 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:25.067172 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebbb69-4552-4232-85b0-d5ef33843f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:25.067787 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebbb69-4552-4232-85b0-d5ef33843f67 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:25.068072 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebbb69-4552-4232-85b0-d5ef33843f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:25.068349 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ebbb69-4552-4232-85b0-d5ef33843f67 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:25.069034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebbb69-4552-4232-85b0-d5ef33843f67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:25.069340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebbb69-4552-4232-85b0-d5ef33843f67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:25.070727 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ebbb69-4552-4232-85b0-d5ef33843f67 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:25.082773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 636/1266] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:25.089001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b7fcf-77b3-4433-899e-142c8934d8c6 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:25.188955 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b7fcf-77b3-4433-899e-142c8934d8c6 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:25.209075 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:25.226523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67c05a-bc59-4164-a635-8799de155162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:25.227338 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67c05a-bc59-4164-a635-8799de155162 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:25.227653 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67c05a-bc59-4164-a635-8799de155162 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:25.227932 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b67c05a-bc59-4164-a635-8799de155162 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:25.235735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67c05a-bc59-4164-a635-8799de155162 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:25.237162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b67c05a-bc59-4164-a635-8799de155162 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:25.248403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-2b67c05a-bc59-4164-a635-8799de155162 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 08:50:25.248403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:25.248403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:25.248403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:25.248403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:25.248403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:25.248403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:25.248403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:25.248403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:25.248403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:25.248403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:25.248403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:25.248403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:50:25.248403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:50:25.248403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:50:25.248403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:50:25.249540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:25.249540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:25.249540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:50:25.249540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:50:25.249540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:50:25.249540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:50:25.249540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:25.249540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:25.249540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:25.249540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:25.249540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:25.249540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:25.249540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:25.249540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:25.249540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:25.249540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:25.249540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:25.249540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:25.250687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:25.250687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:25.250687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:50:25.250687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:50:25.250687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:50:25.250687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:50:25.250687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:50:25.250687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:50:25.250687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 08:50:25.250687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:50:25.250687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 631/1267] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:50:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:25.255599 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebbb69-4552-4232-85b0-d5ef33843f67 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:25.358850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ebbb69-4552-4232-85b0-d5ef33843f67 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:25.392037 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc5344-7d14-409c-8f0b-57310870f905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:25.392648 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc5344-7d14-409c-8f0b-57310870f905 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:25.392929 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc5344-7d14-409c-8f0b-57310870f905 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:25.393201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc5344-7d14-409c-8f0b-57310870f905 None admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:25.394013 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bc5344-7d14-409c-8f0b-57310870f905 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:25.394320 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bc5344-7d14-409c-8f0b-57310870f905 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:25.395741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bc5344-7d14-409c-8f0b-57310870f905 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:25.404651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:25.412877 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 637/1268] 10.222.0.46 () {56 vars in 1143 bytes} [Tue Aug 2 08:50:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:25.418532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67c05a-bc59-4164-a635-8799de155162 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:25.517303 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b67c05a-bc59-4164-a635-8799de155162 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:25.549790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d816e6-f600-4f38-b1f5-76c9b482776b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:25.550454 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d816e6-f600-4f38-b1f5-76c9b482776b None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:25.550735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d816e6-f600-4f38-b1f5-76c9b482776b None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:25.551008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d816e6-f600-4f38-b1f5-76c9b482776b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:25.562594 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d816e6-f600-4f38-b1f5-76c9b482776b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:25.564026 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d816e6-f600-4f38-b1f5-76c9b482776b None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:25.578988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-f3d816e6-f600-4f38-b1f5-76c9b482776b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:25.578988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:25.578988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:25.578988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:25.578988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:25.578988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:25.578988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:25.578988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:25.578988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:25.578988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:25.578988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:25.578988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:25.578988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:50:25.578988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:50:25.578988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:50:25.580057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:50:25.580057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:25.580057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:25.580057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:25.580057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:25.580057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:25.580057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:25.580057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:25.580057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:25.580057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:25.580057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:25.580057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:25.580057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:25.580057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:25.580057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:25.580057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:25.580057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:25.581170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:50:25.581170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:50:25.581170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:25.581170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:50:25.581170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 632/1269] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:50:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:25.587509 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc5344-7d14-409c-8f0b-57310870f905 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:25.692303 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc5344-7d14-409c-8f0b-57310870f905 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:25.726149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b579359-2334-4e16-b09d-1acdcd6944dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:25.726768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b579359-2334-4e16-b09d-1acdcd6944dc None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:25.727048 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b579359-2334-4e16-b09d-1acdcd6944dc None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:25.727321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b579359-2334-4e16-b09d-1acdcd6944dc None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:25.728045 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b579359-2334-4e16-b09d-1acdcd6944dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:25.728332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b579359-2334-4e16-b09d-1acdcd6944dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:25.729685 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b579359-2334-4e16-b09d-1acdcd6944dc None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:25.740791 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 638/1270] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:50:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:25.747278 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d816e6-f600-4f38-b1f5-76c9b482776b None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:25.779544 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:25.847326 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d816e6-f600-4f38-b1f5-76c9b482776b None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:25.881070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06829dbf-8c61-468b-9e8c-acb0873958c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:25.881932 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06829dbf-8c61-468b-9e8c-acb0873958c5 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:25.882381 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06829dbf-8c61-468b-9e8c-acb0873958c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:25.883272 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06829dbf-8c61-468b-9e8c-acb0873958c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:25.889798 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06829dbf-8c61-468b-9e8c-acb0873958c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:25.891293 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06829dbf-8c61-468b-9e8c-acb0873958c5 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:25.902799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-06829dbf-8c61-468b-9e8c-acb0873958c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:50:25.902799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:25.902799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:25.902799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:25.902799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:25.902799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:25.902799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:25.902799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:25.902799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:25.902799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:25.902799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:25.902799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:25.902799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:50:25.902799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:50:25.902799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:50:25.903975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:50:25.903975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:25.903975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:25.903975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:25.903975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:25.903975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:25.903975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:25.903975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:25.903975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:25.903975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:25.903975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:25.903975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:25.903975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:25.903975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:25.903975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:25.903975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:25.903975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:25.904907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:50:25.904907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:50:25.904907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:50:25.904907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:50:25.904907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:50:25.904907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:50:25.904907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:50:25.904907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:50:25.904907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 633/1271] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:50:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:25.909930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b579359-2334-4e16-b09d-1acdcd6944dc None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:25.976449 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:26.019192 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b579359-2334-4e16-b09d-1acdcd6944dc None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:26.053420 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7a0b50-7f89-46b4-880f-adc1a7edd830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:26.054011 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a0b50-7f89-46b4-880f-adc1a7edd830 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:26.054297 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a0b50-7f89-46b4-880f-adc1a7edd830 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:26.054571 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7a0b50-7f89-46b4-880f-adc1a7edd830 None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:26.055341 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a0b50-7f89-46b4-880f-adc1a7edd830 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:26.055647 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a0b50-7f89-46b4-880f-adc1a7edd830 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:26.057740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7a0b50-7f89-46b4-880f-adc1a7edd830 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:26.069692 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 639/1272] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:50:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:26.076032 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06829dbf-8c61-468b-9e8c-acb0873958c5 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:26.165923 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:26.174375 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06829dbf-8c61-468b-9e8c-acb0873958c5 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:26.207517 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07553313-e6e5-49ef-87c3-829e7140f1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:26.208174 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07553313-e6e5-49ef-87c3-829e7140f1b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:26.208459 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07553313-e6e5-49ef-87c3-829e7140f1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:26.208739 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07553313-e6e5-49ef-87c3-829e7140f1b3 None admin] PATH_INFO: `/v3/projects/4227a5b804c0414c99d407bb1415cf49/users/cad5cc9927374836b0cc74db85b686fd/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:26.210697 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07553313-e6e5-49ef-87c3-829e7140f1b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=4227a5b804c0414c99d407bb1415cf49, user_id=cad5cc9927374836b0cc74db85b686fd, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:26.212644 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07553313-e6e5-49ef-87c3-829e7140f1b3 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:26.339047 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 634/1273] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 2 08:50:26 2022] PUT /identity/v3/projects/4227a5b804c0414c99d407bb1415cf49/users/cad5cc9927374836b0cc74db85b686fd/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 08:50:28.336143 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b8739-affe-418c-b506-2ba405141685 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:28.336767 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b8739-affe-418c-b506-2ba405141685 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:28.336767 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403b8739-affe-418c-b506-2ba405141685 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:28.338564 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 640/1274] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:50:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:50:28.343963 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058857cf-2d5b-4de8-a386-9e856b99ba97 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:28.344656 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058857cf-2d5b-4de8-a386-9e856b99ba97 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:28.344656 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058857cf-2d5b-4de8-a386-9e856b99ba97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:28.366160 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-058857cf-2d5b-4de8-a386-9e856b99ba97 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:50:28.367334 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-058857cf-2d5b-4de8-a386-9e856b99ba97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:50:28.434930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 635/1275] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:50:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:50:28.458787 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b8739-affe-418c-b506-2ba405141685 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:28.559925 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403b8739-affe-418c-b506-2ba405141685 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:28.577559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:28.593691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0283b06-a57f-4975-b0eb-16962978adc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:28.594329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0283b06-a57f-4975-b0eb-16962978adc2 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:28.594628 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0283b06-a57f-4975-b0eb-16962978adc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:28.594901 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0283b06-a57f-4975-b0eb-16962978adc2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:28.604957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0283b06-a57f-4975-b0eb-16962978adc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 08:50:28.610203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-c0283b06-a57f-4975-b0eb-16962978adc2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:50:28.610203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:28.610203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:28.610203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:28.610203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:28.610203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:28.610203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:28.610203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:28.610203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:28.610203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:28.610203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:28.610203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:28.610203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 08:50:28.610203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 08:50:28.610203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 08:50:28.610203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 08:50:28.611353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 08:50:28.611353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 08:50:28.611353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 08:50:28.611353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 08:50:28.611353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:28.611353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:28.611353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:28.611353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:28.611353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:28.611353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:28.611353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:28.611353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:28.611353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:28.611353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:28.611353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:28.611353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:28.612233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:28.612233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:28.612233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:28.612233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:28.612233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 08:50:28.612233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 08:50:28.612233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 08:50:28.612233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 08:50:28.612233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 08:50:28.612233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 08:50:28.612233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 08:50:28.612233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:50:28.612233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 641/1276] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 2 08:50:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:28.617261 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058857cf-2d5b-4de8-a386-9e856b99ba97 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:28.703005 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:28.720309 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058857cf-2d5b-4de8-a386-9e856b99ba97 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:28.753293 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7a11b-1e63-48e3-9bb2-8ee350fd2074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:28.753924 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7a11b-1e63-48e3-9bb2-8ee350fd2074 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:28.754214 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7a11b-1e63-48e3-9bb2-8ee350fd2074 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:28.754493 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7a11b-1e63-48e3-9bb2-8ee350fd2074 None admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:28.755201 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d7a11b-1e63-48e3-9bb2-8ee350fd2074 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:28.755512 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d7a11b-1e63-48e3-9bb2-8ee350fd2074 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:28.756876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d7a11b-1e63-48e3-9bb2-8ee350fd2074 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:28.765229 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 636/1277] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 2 08:50:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:28.770962 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0283b06-a57f-4975-b0eb-16962978adc2 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:28.873678 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0283b06-a57f-4975-b0eb-16962978adc2 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:28.906549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6f00f-eea2-4f7c-bbb1-3392991763bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:28.907188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6f00f-eea2-4f7c-bbb1-3392991763bc None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:28.907473 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6f00f-eea2-4f7c-bbb1-3392991763bc None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:28.907754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d6f00f-eea2-4f7c-bbb1-3392991763bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:28.918074 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d6f00f-eea2-4f7c-bbb1-3392991763bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:28.919514 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d6f00f-eea2-4f7c-bbb1-3392991763bc None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:28.934820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-b6d6f00f-eea2-4f7c-bbb1-3392991763bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:28.934820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:28.934820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:28.934820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:28.934820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:28.934820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:28.934820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:28.934820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:28.934820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:28.934820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:28.934820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:28.934820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:28.934820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:50:28.934820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:50:28.934820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:50:28.935940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:50:28.935940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:28.935940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:28.935940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:28.935940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:28.935940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:28.935940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:28.935940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:28.935940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:28.935940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:28.935940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:28.935940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:28.935940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:28.935940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:28.935940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:28.935940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:28.935940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:28.936884 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:50:28.936884 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:50:28.936884 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:28.936884 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:50:28.936884 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 642/1278] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:50:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 08:50:28.941692 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7a11b-1e63-48e3-9bb2-8ee350fd2074 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:29.039161 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7a11b-1e63-48e3-9bb2-8ee350fd2074 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:29.073044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c64d7c4-2004-475b-bde6-bf12597bdca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:29.073655 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64d7c4-2004-475b-bde6-bf12597bdca1 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:29.073968 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64d7c4-2004-475b-bde6-bf12597bdca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:29.074248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64d7c4-2004-475b-bde6-bf12597bdca1 None admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:29.075283 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c64d7c4-2004-475b-bde6-bf12597bdca1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:29.075592 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c64d7c4-2004-475b-bde6-bf12597bdca1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:29.076952 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c64d7c4-2004-475b-bde6-bf12597bdca1 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:29.088426 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 637/1279] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:29.094097 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6f00f-eea2-4f7c-bbb1-3392991763bc None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:29.125047 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:29.191482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d6f00f-eea2-4f7c-bbb1-3392991763bc None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:29.224235 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe16ba-f7d7-4b61-a497-f87ea5c8d2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:29.224864 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe16ba-f7d7-4b61-a497-f87ea5c8d2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:29.225142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe16ba-f7d7-4b61-a497-f87ea5c8d2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:29.225414 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fe16ba-f7d7-4b61-a497-f87ea5c8d2b9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:29.233155 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe16ba-f7d7-4b61-a497-f87ea5c8d2b9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:29.234608 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fe16ba-f7d7-4b61-a497-f87ea5c8d2b9 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:29.246916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-c8fe16ba-f7d7-4b61-a497-f87ea5c8d2b9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 08:50:29.246916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:29.246916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:29.246916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:29.246916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:29.246916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:29.246916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:29.246916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:29.246916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:29.246916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:29.246916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:29.246916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:29.246916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 08:50:29.246916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 08:50:29.246916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 08:50:29.246916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 08:50:29.253486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:29.253486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:29.253486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 08:50:29.253486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:50:29.253486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 08:50:29.253486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 08:50:29.253486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:29.253486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:29.253486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:29.253486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:29.253486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:29.253486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:29.253486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:29.253486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:29.253486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:29.253486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:29.253486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:29.253486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:29.254702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:29.254702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:29.254702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 08:50:29.254702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 08:50:29.254702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 08:50:29.254702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 08:50:29.254702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 08:50:29.254702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 08:50:29.254702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 08:50:29.254702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:50:29.254702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 643/1280] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:50:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:29.255393 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c64d7c4-2004-475b-bde6-bf12597bdca1 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:29.258120 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:29.351504 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c64d7c4-2004-475b-bde6-bf12597bdca1 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:29.383714 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf7f8e-d1aa-47d5-a687-423bf653f04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:29.384807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf7f8e-d1aa-47d5-a687-423bf653f04e None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:29.385082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf7f8e-d1aa-47d5-a687-423bf653f04e None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:29.385353 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf7f8e-d1aa-47d5-a687-423bf653f04e None admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:29.386155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf7f8e-d1aa-47d5-a687-423bf653f04e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:29.386456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf7f8e-d1aa-47d5-a687-423bf653f04e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:29.387858 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf7f8e-d1aa-47d5-a687-423bf653f04e None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:29.405058 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 638/1281] 10.222.0.46 () {56 vars in 1143 bytes} [Tue Aug 2 08:50:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:50:29.410793 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe16ba-f7d7-4b61-a497-f87ea5c8d2b9 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:29.500881 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:29.510388 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fe16ba-f7d7-4b61-a497-f87ea5c8d2b9 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:29.542905 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c45b6-04c8-4867-a24d-4fbc9b3b3473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:29.543508 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c45b6-04c8-4867-a24d-4fbc9b3b3473 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:29.543783 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c45b6-04c8-4867-a24d-4fbc9b3b3473 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:29.544053 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c45b6-04c8-4867-a24d-4fbc9b3b3473 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:29.609012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c45b6-04c8-4867-a24d-4fbc9b3b3473 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:29.610573 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889c45b6-04c8-4867-a24d-4fbc9b3b3473 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:29.698215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:29.708237 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 644/1282] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 2 08:50:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 08:50:29.714111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf7f8e-d1aa-47d5-a687-423bf653f04e None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:29.778579 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:29.824307 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf7f8e-d1aa-47d5-a687-423bf653f04e None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:29.857266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd16ff3-0720-47f5-8db7-57a43576fe30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:29.857948 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd16ff3-0720-47f5-8db7-57a43576fe30 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:29.858235 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd16ff3-0720-47f5-8db7-57a43576fe30 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:29.858511 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd16ff3-0720-47f5-8db7-57a43576fe30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:29.870504 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd16ff3-0720-47f5-8db7-57a43576fe30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:29.871950 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd16ff3-0720-47f5-8db7-57a43576fe30 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:29.886747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-2bd16ff3-0720-47f5-8db7-57a43576fe30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:29.886747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:29.886747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:29.886747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:29.886747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:29.886747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:29.886747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:29.886747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:29.886747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:29.886747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:29.886747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:29.886747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:29.886747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 08:50:29.886747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 08:50:29.886747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 08:50:29.887868 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 08:50:29.887868 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:29.887868 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:29.887868 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:29.887868 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:29.887868 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:29.887868 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:29.887868 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:29.887868 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:29.887868 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:29.887868 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:29.887868 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:29.887868 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:29.887868 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:29.887868 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:29.887868 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:29.887868 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:29.888985 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 08:50:29.888985 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 08:50:29.888985 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 08:50:29.888985 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:50:29.888985 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 639/1283] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 08:50:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:29.893621 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c45b6-04c8-4867-a24d-4fbc9b3b3473 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:29.991852 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c45b6-04c8-4867-a24d-4fbc9b3b3473 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:30.024216 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53541c19-8c0a-412e-84b6-98c12c9865e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:30.025468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53541c19-8c0a-412e-84b6-98c12c9865e7 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:30.025906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53541c19-8c0a-412e-84b6-98c12c9865e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:30.026336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53541c19-8c0a-412e-84b6-98c12c9865e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:30.027053 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53541c19-8c0a-412e-84b6-98c12c9865e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:30.027361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53541c19-8c0a-412e-84b6-98c12c9865e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:30.028717 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53541c19-8c0a-412e-84b6-98c12c9865e7 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:30.039713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 645/1284] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 2 08:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:30.045873 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd16ff3-0720-47f5-8db7-57a43576fe30 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:30.146683 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd16ff3-0720-47f5-8db7-57a43576fe30 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:30.160237 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:30.179794 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663806d2-a17c-4071-ac77-a3d5771c0df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:30.180421 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663806d2-a17c-4071-ac77-a3d5771c0df6 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:30.180697 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663806d2-a17c-4071-ac77-a3d5771c0df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:30.181005 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663806d2-a17c-4071-ac77-a3d5771c0df6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:30.187448 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663806d2-a17c-4071-ac77-a3d5771c0df6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:30.188920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663806d2-a17c-4071-ac77-a3d5771c0df6 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:30.200217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application [None req-663806d2-a17c-4071-ac77-a3d5771c0df6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:50:30.200217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:30.200217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:30.200217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:30.200217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:30.200217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:30.200217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:30.200217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:30.200217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:30.200217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:30.200217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:30.200217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:30.200217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 08:50:30.200217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 08:50:30.200217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 08:50:30.201315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 08:50:30.201315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:30.201315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:30.201315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:30.201315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:30.201315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:30.201315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:30.201315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:30.201315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:30.201315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:30.201315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:30.201315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:30.201315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:30.201315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:30.201315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:30.201315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:30.201315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:30.207355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 08:50:30.207355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 08:50:30.207355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 08:50:30.207355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 08:50:30.207355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 08:50:30.207355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 08:50:30.207355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 08:50:30.207355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: ERROR keystone.server.flask.application Aug 02 08:50:30.207355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 640/1285] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 2 08:50:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:30.207984 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53541c19-8c0a-412e-84b6-98c12c9865e7 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:30.257233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:30.305992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53541c19-8c0a-412e-84b6-98c12c9865e7 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:30.340600 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b9bf7-78d0-4abe-9abc-81dd1f34f82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:30.341242 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b9bf7-78d0-4abe-9abc-81dd1f34f82b None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:30.341524 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b9bf7-78d0-4abe-9abc-81dd1f34f82b None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:30.341799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b9bf7-78d0-4abe-9abc-81dd1f34f82b None admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:30.342576 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b9bf7-78d0-4abe-9abc-81dd1f34f82b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:30.342878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b9bf7-78d0-4abe-9abc-81dd1f34f82b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:30.344305 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b9bf7-78d0-4abe-9abc-81dd1f34f82b None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:30.356360 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 646/1286] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 2 08:50:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:30.362772 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663806d2-a17c-4071-ac77-a3d5771c0df6 None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:30.460473 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663806d2-a17c-4071-ac77-a3d5771c0df6 None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:30.493405 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e96b3-df0c-43b5-9f04-a49bad7fb010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:30.494039 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e96b3-df0c-43b5-9f04-a49bad7fb010 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:30.494322 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e96b3-df0c-43b5-9f04-a49bad7fb010 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:30.494595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e96b3-df0c-43b5-9f04-a49bad7fb010 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:30.495367 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e96b3-df0c-43b5-9f04-a49bad7fb010 None admin] RBAC: Adding query filter params (role.id=8ce0850e15804f3d811e88cc9841e673, scope.project.id=4227a5b804c0414c99d407bb1415cf49, user.id=cad5cc9927374836b0cc74db85b686fd) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:30.495674 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e96b3-df0c-43b5-9f04-a49bad7fb010 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:30.497081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e96b3-df0c-43b5-9f04-a49bad7fb010 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:30.502716 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 641/1287] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 2 08:50:30 2022] GET /identity/v3/role_assignments?user.id=cad5cc9927374836b0cc74db85b686fd&scope.project.id=4227a5b804c0414c99d407bb1415cf49&role.id=8ce0850e15804f3d811e88cc9841e673 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:32.551553 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cbd50-fa11-4218-b0f8-da8f475c0355 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:32.552138 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cbd50-fa11-4218-b0f8-da8f475c0355 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:32.552138 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552cbd50-fa11-4218-b0f8-da8f475c0355 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:32.553411 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 647/1288] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:50:32.558851 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e9889-3994-4e6c-9751-8b31a5f68f1b None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:32.559229 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e9889-3994-4e6c-9751-8b31a5f68f1b None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:32.559403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e9889-3994-4e6c-9751-8b31a5f68f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:32.577837 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-307e9889-3994-4e6c-9751-8b31a5f68f1b None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:50:32.579107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-307e9889-3994-4e6c-9751-8b31a5f68f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:50:32.608247 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:32.643983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 642/1289] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:50:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:50:32.666279 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552cbd50-fa11-4218-b0f8-da8f475c0355 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:32.766528 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552cbd50-fa11-4218-b0f8-da8f475c0355 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:32.777513 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:32.800052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ccc555-6df5-4351-864a-40fcedaa8acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:32.800688 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccc555-6df5-4351-864a-40fcedaa8acd None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:32.800965 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccc555-6df5-4351-864a-40fcedaa8acd None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:32.801239 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccc555-6df5-4351-864a-40fcedaa8acd None admin] PATH_INFO: `/v3/services/placement` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:32.801910 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ccc555-6df5-4351-864a-40fcedaa8acd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:32.803293 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ccc555-6df5-4351-864a-40fcedaa8acd None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:32.812055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-f0ccc555-6df5-4351-864a-40fcedaa8acd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 08:50:32.812055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:32.812055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:50:32.812055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:50:32.812055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:50:32.812055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:50:32.812055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:50:32.812055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:50:32.812055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 08:50:32.812055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:50:32.812055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:50:32.812055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:50:32.812055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:32.812055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:32.812055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:32.812055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:32.812055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:32.813289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:32.813289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:32.813289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:32.813289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:32.813289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:32.813289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:32.813289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:50:32.813289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:50:32.813289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:50:32.813289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:50:32.813289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:32.813289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:32.813289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:32.813289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:32.813289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:32.813289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:32.814470 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:32.814470 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:32.814470 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:32.814470 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:32.814470 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:32.814470 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:32.814470 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:32.814470 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:32.814470 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:32.814470 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:32.814470 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:50:32.814470 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:50:32.814470 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 08:50:32.814470 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:50:32.814470 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 648/1290] 10.222.0.46 () {56 vars in 1117 bytes} [Tue Aug 2 08:50:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:32.819253 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e9889-3994-4e6c-9751-8b31a5f68f1b None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:32.919216 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e9889-3994-4e6c-9751-8b31a5f68f1b None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:32.954872 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e509bd8e-00c5-4eb8-a682-f968a48af97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:32.955488 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509bd8e-00c5-4eb8-a682-f968a48af97b None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:32.955769 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509bd8e-00c5-4eb8-a682-f968a48af97b None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:32.956043 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509bd8e-00c5-4eb8-a682-f968a48af97b None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:32.956710 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e509bd8e-00c5-4eb8-a682-f968a48af97b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:32.957012 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e509bd8e-00c5-4eb8-a682-f968a48af97b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:32.958464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e509bd8e-00c5-4eb8-a682-f968a48af97b None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:32.963221 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 643/1291] 10.222.0.46 () {56 vars in 1116 bytes} [Tue Aug 2 08:50:32 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:32.969098 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ccc555-6df5-4351-864a-40fcedaa8acd None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:33.067852 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ccc555-6df5-4351-864a-40fcedaa8acd None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:33.100664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef9854-65e3-4ff7-8906-ba27b7caffe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:33.101271 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef9854-65e3-4ff7-8906-ba27b7caffe5 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:33.102653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef9854-65e3-4ff7-8906-ba27b7caffe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:33.102653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef9854-65e3-4ff7-8906-ba27b7caffe5 None admin] PATH_INFO: `/v3/services` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:33.102653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef9854-65e3-4ff7-8906-ba27b7caffe5 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:33.102971 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef9854-65e3-4ff7-8906-ba27b7caffe5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:33.104212 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cef9854-65e3-4ff7-8906-ba27b7caffe5 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:33.108484 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 649/1292] 10.222.0.46 () {56 vars in 1116 bytes} [Tue Aug 2 08:50:32 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:35.088248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb660031-f167-482e-b7b4-7ad652c7bdc9 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:35.088811 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb660031-f167-482e-b7b4-7ad652c7bdc9 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:35.088811 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb660031-f167-482e-b7b4-7ad652c7bdc9 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:35.090185 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 644/1293] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:50:35.095202 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bd687-717c-4570-828b-9d26807653a9 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:35.095645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bd687-717c-4570-828b-9d26807653a9 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:35.095752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bd687-717c-4570-828b-9d26807653a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:35.114073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-6e1bd687-717c-4570-828b-9d26807653a9 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:50:35.115409 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6e1bd687-717c-4570-828b-9d26807653a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:50:35.144759 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:35.180467 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 650/1294] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:50:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:50:35.204587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb660031-f167-482e-b7b4-7ad652c7bdc9 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:35.235132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:35.303802 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb660031-f167-482e-b7b4-7ad652c7bdc9 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:35.336825 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b68bd3-46ce-4178-b1ce-2659cde7a2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:35.338111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b68bd3-46ce-4178-b1ce-2659cde7a2c6 None admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:35.338389 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b68bd3-46ce-4178-b1ce-2659cde7a2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:35.338656 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b68bd3-46ce-4178-b1ce-2659cde7a2c6 None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:35.339562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b68bd3-46ce-4178-b1ce-2659cde7a2c6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:35.341276 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b68bd3-46ce-4178-b1ce-2659cde7a2c6 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:35.424405 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 645/1295] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 2 08:50:35 2022] POST /identity/v3/services => generated 246 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:50:37.416762 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef73eae-c2bd-45ca-876b-5cd8b3de271d None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:37.417319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef73eae-c2bd-45ca-876b-5cd8b3de271d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:37.417319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef73eae-c2bd-45ca-876b-5cd8b3de271d None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:37.418731 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 651/1296] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:50:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:50:37.423915 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de206a6-be85-4b0e-bd42-42cfbefd612d None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:37.424324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de206a6-be85-4b0e-bd42-42cfbefd612d None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:37.424495 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de206a6-be85-4b0e-bd42-42cfbefd612d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:37.442803 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-9de206a6-be85-4b0e-bd42-42cfbefd612d None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:50:37.443974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-9de206a6-be85-4b0e-bd42-42cfbefd612d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:50:37.516127 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 646/1297] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:50:37 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:50:37.538659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef73eae-c2bd-45ca-876b-5cd8b3de271d None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:37.639146 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef73eae-c2bd-45ca-876b-5cd8b3de271d None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:37.671573 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a39b7-1db4-4221-958b-7f40cbc61efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:37.672138 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a39b7-1db4-4221-958b-7f40cbc61efe None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:37.672414 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a39b7-1db4-4221-958b-7f40cbc61efe None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:37.672681 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2a39b7-1db4-4221-958b-7f40cbc61efe None admin] PATH_INFO: `/v3/services/placement` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:37.673291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2a39b7-1db4-4221-958b-7f40cbc61efe None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:37.674851 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2a39b7-1db4-4221-958b-7f40cbc61efe None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:37.683887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-0b2a39b7-1db4-4221-958b-7f40cbc61efe None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 08:50:37.683887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:37.683887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:50:37.683887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:50:37.683887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:50:37.683887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:50:37.683887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:50:37.683887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:50:37.683887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 08:50:37.683887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:50:37.683887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:50:37.683887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:50:37.683887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:37.683887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:37.683887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:37.683887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:37.683887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:37.685064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:37.685064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:37.685064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:37.685064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:37.685064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:37.685064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:37.685064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:50:37.685064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:50:37.685064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:50:37.685064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:50:37.685064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:37.685064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:37.685064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:37.685064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:37.685064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:37.685064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:37.686394 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:37.686394 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:37.686394 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:37.686394 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:37.686394 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:37.686394 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:37.686394 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:37.686394 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:37.686394 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:37.686394 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:37.686394 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:50:37.686394 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:50:37.686394 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 08:50:37.686394 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:50:37.686394 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 652/1298] 10.222.0.46 () {56 vars in 1117 bytes} [Tue Aug 2 08:50:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:37.691095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de206a6-be85-4b0e-bd42-42cfbefd612d None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:37.709821 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:37.792706 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de206a6-be85-4b0e-bd42-42cfbefd612d None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:37.826920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd66ad-564a-4524-ae50-0bd8bcc5bf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:37.827539 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd66ad-564a-4524-ae50-0bd8bcc5bf6d None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:37.827824 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd66ad-564a-4524-ae50-0bd8bcc5bf6d None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:37.828100 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd66ad-564a-4524-ae50-0bd8bcc5bf6d None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:37.828774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd66ad-564a-4524-ae50-0bd8bcc5bf6d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:37.829129 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd66ad-564a-4524-ae50-0bd8bcc5bf6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:37.830535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd66ad-564a-4524-ae50-0bd8bcc5bf6d None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:37.835686 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 647/1299] 10.222.0.46 () {56 vars in 1116 bytes} [Tue Aug 2 08:50:37 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:37.841727 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a39b7-1db4-4221-958b-7f40cbc61efe None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:37.873156 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:37.939641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2a39b7-1db4-4221-958b-7f40cbc61efe None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:37.973016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f9658-1df7-4b6f-b625-e8b53bcf62af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:37.973627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f9658-1df7-4b6f-b625-e8b53bcf62af None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:37.973930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f9658-1df7-4b6f-b625-e8b53bcf62af None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:37.974210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f9658-1df7-4b6f-b625-e8b53bcf62af None admin] PATH_INFO: `/v3/endpoints` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:37.974962 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f9658-1df7-4b6f-b625-e8b53bcf62af None admin] RBAC: Adding query filter params (interface=public, service_id=c09e8ac4528d4ab1a085851b9170fcd3, region_id=RegionOne) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:37.975274 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f9658-1df7-4b6f-b625-e8b53bcf62af None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:37.976659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f9658-1df7-4b6f-b625-e8b53bcf62af None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:37.982077 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 653/1300] 10.222.0.46 () {56 vars in 1251 bytes} [Tue Aug 2 08:50:37 2022] GET /identity/v3/endpoints?service_id=c09e8ac4528d4ab1a085851b9170fcd3&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 08:50:37.987792 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd66ad-564a-4524-ae50-0bd8bcc5bf6d None admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:38.071259 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:38.088981 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd66ad-564a-4524-ae50-0bd8bcc5bf6d None admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:38.123441 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078f1709-137f-42cd-aab7-1fb29264220b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:38.124059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f1709-137f-42cd-aab7-1fb29264220b None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:38.124342 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f1709-137f-42cd-aab7-1fb29264220b None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:38.124619 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f1709-137f-42cd-aab7-1fb29264220b None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:38.125407 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f1709-137f-42cd-aab7-1fb29264220b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:38.126791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078f1709-137f-42cd-aab7-1fb29264220b None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:38.133563 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 648/1301] 10.222.0.46 () {56 vars in 1087 bytes} [Tue Aug 2 08:50:37 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 08:50:40.150101 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb153e9-b165-43d8-9888-5a701aa532b0 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:40.150649 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb153e9-b165-43d8-9888-5a701aa532b0 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:40.150649 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb153e9-b165-43d8-9888-5a701aa532b0 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:40.152068 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 654/1302] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:50:40.157391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62604234-563d-4d80-aeb6-79469160c166 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:40.157768 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62604234-563d-4d80-aeb6-79469160c166 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:40.157973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62604234-563d-4d80-aeb6-79469160c166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:40.176177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-62604234-563d-4d80-aeb6-79469160c166 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:50:40.177354 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-62604234-563d-4d80-aeb6-79469160c166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:50:40.244250 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 649/1303] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:50:40 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 08:50:40.266660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb153e9-b165-43d8-9888-5a701aa532b0 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:40.371291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:40.379160 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb153e9-b165-43d8-9888-5a701aa532b0 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:40.411048 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a4f6c-6e12-4b74-b0a9-1b9deb1c6c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:40.411678 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a4f6c-6e12-4b74-b0a9-1b9deb1c6c89 None admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:40.412021 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a4f6c-6e12-4b74-b0a9-1b9deb1c6c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:40.412335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a4f6c-6e12-4b74-b0a9-1b9deb1c6c89 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:40.412965 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a4f6c-6e12-4b74-b0a9-1b9deb1c6c89 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:40.414356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866a4f6c-6e12-4b74-b0a9-1b9deb1c6c89 None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:40.424784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application [None req-866a4f6c-6e12-4b74-b0a9-1b9deb1c6c89 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 08:50:40.424784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:40.424784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 08:50:40.424784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 08:50:40.424784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 08:50:40.424784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 08:50:40.424784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 08:50:40.424784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:50:40.424784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 08:50:40.424784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:50:40.424784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 08:50:40.424784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:50:40.424784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 08:50:40.424784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 08:50:40.424784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 08:50:40.424784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 08:50:40.424784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 08:50:40.425990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 08:50:40.425990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 08:50:40.425990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 08:50:40.425990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 08:50:40.425990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 08:50:40.425990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 08:50:40.425990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 08:50:40.425990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 08:50:40.425990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 08:50:40.425990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 08:50:40.425990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 08:50:40.425990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 08:50:40.425990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 08:50:40.425990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 08:50:40.425990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 08:50:40.425990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 08:50:40.427073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 08:50:40.427073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application with Lock( Aug 02 08:50:40.427073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 08:50:40.427073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self._enter() Aug 02 08:50:40.427073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 08:50:40.427073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 08:50:40.427073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 08:50:40.427073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application return self.creator() Aug 02 08:50:40.427073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 08:50:40.427073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application created_value = creator( Aug 02 08:50:40.427073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 08:50:40.427073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 08:50:40.427073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 08:50:40.427073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: ERROR keystone.server.flask.application Aug 02 08:50:40.427073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 655/1304] 10.222.0.46 () {56 vars in 1117 bytes} [Tue Aug 2 08:50:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 08:50:40.432844 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62604234-563d-4d80-aeb6-79469160c166 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:40.507375 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:40.530101 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62604234-563d-4d80-aeb6-79469160c166 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:40.563885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d1607-0e5d-4bb5-9410-9763d95966f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:40.564546 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d1607-0e5d-4bb5-9410-9763d95966f5 None admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:40.564829 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d1607-0e5d-4bb5-9410-9763d95966f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:40.565104 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d1607-0e5d-4bb5-9410-9763d95966f5 None admin] PATH_INFO: `/v3/services` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:40.565770 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d1607-0e5d-4bb5-9410-9763d95966f5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 08:50:40.566101 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d1607-0e5d-4bb5-9410-9763d95966f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:40.567452 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d1607-0e5d-4bb5-9410-9763d95966f5 None admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:40.572486 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 650/1305] 10.222.0.46 () {56 vars in 1116 bytes} [Tue Aug 2 08:50:40 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 08:50:40.578581 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a4f6c-6e12-4b74-b0a9-1b9deb1c6c89 None admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:50:40.679017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a4f6c-6e12-4b74-b0a9-1b9deb1c6c89 None admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:50:40.711487 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8ffd7d-751b-4020-b9af-cdeb35e7a9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:50:40.712636 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ffd7d-751b-4020-b9af-cdeb35e7a9bc None admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:50:40.715278 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ffd7d-751b-4020-b9af-cdeb35e7a9bc None admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:50:40.715278 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8ffd7d-751b-4020-b9af-cdeb35e7a9bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:50:40.715278 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ffd7d-751b-4020-b9af-cdeb35e7a9bc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:50:40.715520 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8ffd7d-751b-4020-b9af-cdeb35e7a9bc None admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:50:40.841609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 656/1306] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 08:50:40 2022] POST /identity/v3/endpoints => generated 331 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 08:50:50.624724 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=30 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:50:50.836008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:21.084037 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a55c3-49ce-4177-9a01-d045bce5e3fd None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:21.084631 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a55c3-49ce-4177-9a01-d045bce5e3fd None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:21.085032 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a55c3-49ce-4177-9a01-d045bce5e3fd None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:21.087513 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 651/1307] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:57:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:57:21.094718 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18af68f-204f-4102-b395-d6283838d3dc None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:21.095568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18af68f-204f-4102-b395-d6283838d3dc None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:21.095568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18af68f-204f-4102-b395-d6283838d3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:21.211224 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-e18af68f-204f-4102-b395-d6283838d3dc None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:21.212666 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e18af68f-204f-4102-b395-d6283838d3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:21.362752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 657/1308] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:57:21.425969 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c711979-1295-4556-9c29-55899d884238 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:21.426285 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c711979-1295-4556-9c29-55899d884238 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:21.426564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c711979-1295-4556-9c29-55899d884238 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:21.427955 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 652/1309] 10.222.0.46 () {54 vars in 965 bytes} [Tue Aug 2 08:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:57:21.437695 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9f56c-2396-4314-bf0f-46f83f82e523 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:21.438529 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9f56c-2396-4314-bf0f-46f83f82e523 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:21.439149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9f56c-2396-4314-bf0f-46f83f82e523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:21.565799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-b9e9f56c-2396-4314-bf0f-46f83f82e523 None None] MFA Rules not processed for user `ddf6f4d84bb34d2f810a0f6cec36da4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:21.567439 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b9e9f56c-2396-4314-bf0f-46f83f82e523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:21.595069 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:21.662440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 658/1310] 10.222.0.46 () {58 vars in 1062 bytes} [Tue Aug 2 08:57:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:57:21.671697 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c711979-1295-4556-9c29-55899d884238 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:21.673469 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-8c711979-1295-4556-9c29-55899d884238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:21.842438 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c711979-1295-4556-9c29-55899d884238 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:21.956626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f71769-aea1-4bc3-9dde-eb571c5a1917 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:21.957440 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f71769-aea1-4bc3-9dde-eb571c5a1917 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:21.957842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f71769-aea1-4bc3-9dde-eb571c5a1917 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:21.958151 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f71769-aea1-4bc3-9dde-eb571c5a1917 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:21.960244 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-05f71769-aea1-4bc3-9dde-eb571c5a1917 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:22.032547 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:22.102643 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f71769-aea1-4bc3-9dde-eb571c5a1917 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:22.104459 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f71769-aea1-4bc3-9dde-eb571c5a1917 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:22.128585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 653/1311] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:57:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:57:24.600671 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89b999-6810-4a72-8c03-e328c2d56274 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:24.601204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89b999-6810-4a72-8c03-e328c2d56274 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:24.601204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89b999-6810-4a72-8c03-e328c2d56274 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:24.602757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 659/1312] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:57:24.607994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685d921-f716-4b8e-8168-d439c742d829 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:24.608425 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685d921-f716-4b8e-8168-d439c742d829 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:24.608573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685d921-f716-4b8e-8168-d439c742d829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:24.701514 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-1685d921-f716-4b8e-8168-d439c742d829 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:24.702771 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-1685d921-f716-4b8e-8168-d439c742d829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:24.800925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 654/1313] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:57:24.840890 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89b999-6810-4a72-8c03-e328c2d56274 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:24.842481 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4e89b999-6810-4a72-8c03-e328c2d56274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:24.948631 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89b999-6810-4a72-8c03-e328c2d56274 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:25.022232 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:25.024877 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bbcc1c-ca8a-4e9f-ba46-14717342302b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:25.026022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbcc1c-ca8a-4e9f-ba46-14717342302b service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:25.026303 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbcc1c-ca8a-4e9f-ba46-14717342302b service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:25.026621 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bbcc1c-ca8a-4e9f-ba46-14717342302b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:25.028308 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f9bbcc1c-ca8a-4e9f-ba46-14717342302b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:25.143825 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bbcc1c-ca8a-4e9f-ba46-14717342302b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:25.145281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bbcc1c-ca8a-4e9f-ba46-14717342302b service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:25.164424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 660/1314] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:57:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:57:27.687356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395c059-fb95-47ae-b48c-e204b7d202b6 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:27.687904 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395c059-fb95-47ae-b48c-e204b7d202b6 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:27.688036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395c059-fb95-47ae-b48c-e204b7d202b6 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:27.689375 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 655/1315] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:57:27.694878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a42dc-9dff-4a2f-8767-298518a2d18e None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:27.695320 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a42dc-9dff-4a2f-8767-298518a2d18e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:27.695432 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a42dc-9dff-4a2f-8767-298518a2d18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:27.771494 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-e68a42dc-9dff-4a2f-8767-298518a2d18e None None] MFA Rules not processed for user `7e1257a546984dca904f536574103621`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:27.772892 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e68a42dc-9dff-4a2f-8767-298518a2d18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:27.830571 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:27.880048 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 661/1316] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:57:27 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:57:27.921195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395c059-fb95-47ae-b48c-e204b7d202b6 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:27.922833 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f395c059-fb95-47ae-b48c-e204b7d202b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:27.933577 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:28.034898 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395c059-fb95-47ae-b48c-e204b7d202b6 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:28.124675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2f228a-882f-4dbb-845a-9b3b2bd4bffb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:28.125382 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f228a-882f-4dbb-845a-9b3b2bd4bffb service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:28.125715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f228a-882f-4dbb-845a-9b3b2bd4bffb service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:28.126069 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2f228a-882f-4dbb-845a-9b3b2bd4bffb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:28.128806 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-ec2f228a-882f-4dbb-845a-9b3b2bd4bffb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:28.197940 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:28.246325 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f228a-882f-4dbb-845a-9b3b2bd4bffb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:28.247759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2f228a-882f-4dbb-845a-9b3b2bd4bffb service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:28.271801 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 656/1317] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:57:27 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:57:31.615833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f3906-a553-4828-9f05-9ce5d3694ba0 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:31.616492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f3906-a553-4828-9f05-9ce5d3694ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:31.616492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6f3906-a553-4828-9f05-9ce5d3694ba0 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:31.617742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 662/1318] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:57:31.623484 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42414b98-7d65-40ce-8043-2171a96ef5c9 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:31.624151 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42414b98-7d65-40ce-8043-2171a96ef5c9 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:31.624440 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42414b98-7d65-40ce-8043-2171a96ef5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:31.699739 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-42414b98-7d65-40ce-8043-2171a96ef5c9 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:31.701217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-42414b98-7d65-40ce-8043-2171a96ef5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:31.806576 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 657/1319] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:57:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:57:31.846202 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f3906-a553-4828-9f05-9ce5d3694ba0 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:31.847760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5f6f3906-a553-4828-9f05-9ce5d3694ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:31.956491 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6f3906-a553-4828-9f05-9ce5d3694ba0 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:32.031361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f79193-5497-4dbf-8d09-718e5afe1a79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:32.031986 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f79193-5497-4dbf-8d09-718e5afe1a79 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:32.032272 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f79193-5497-4dbf-8d09-718e5afe1a79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:32.032550 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f79193-5497-4dbf-8d09-718e5afe1a79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:32.034269 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-95f79193-5497-4dbf-8d09-718e5afe1a79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:32.041724 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:32.143508 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f79193-5497-4dbf-8d09-718e5afe1a79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:32.144954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f79193-5497-4dbf-8d09-718e5afe1a79 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:32.166722 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 663/1320] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:57:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:57:35.470725 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f90ee-0187-49ae-a3ee-9f8e998488d7 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:35.471447 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f90ee-0187-49ae-a3ee-9f8e998488d7 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:35.471447 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1f90ee-0187-49ae-a3ee-9f8e998488d7 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:35.472906 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 658/1321] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:57:35.480691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b1725-e3fc-4319-a086-f9569591abaa None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:35.481708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b1725-e3fc-4319-a086-f9569591abaa None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:35.482351 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b1725-e3fc-4319-a086-f9569591abaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:35.583904 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-be4b1725-e3fc-4319-a086-f9569591abaa None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:35.585148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-be4b1725-e3fc-4319-a086-f9569591abaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:35.678650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:35.739690 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 664/1322] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:57:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:57:35.779376 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f90ee-0187-49ae-a3ee-9f8e998488d7 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:35.781436 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-3f1f90ee-0187-49ae-a3ee-9f8e998488d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:35.802223 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:35.912068 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1f90ee-0187-49ae-a3ee-9f8e998488d7 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:36.048828 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3aeb9-7791-48cf-91b7-d662b28b7a4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:36.049822 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3aeb9-7791-48cf-91b7-d662b28b7a4f service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:36.050304 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3aeb9-7791-48cf-91b7-d662b28b7a4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:36.050763 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3aeb9-7791-48cf-91b7-d662b28b7a4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:36.054171 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-a4e3aeb9-7791-48cf-91b7-d662b28b7a4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:36.153579 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:36.209917 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e3aeb9-7791-48cf-91b7-d662b28b7a4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:36.212740 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e3aeb9-7791-48cf-91b7-d662b28b7a4f service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:36.249872 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 659/1323] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:57:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:57:38.415571 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b4ba1-2493-43d2-954b-e2c4bd54243d None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:38.416783 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b4ba1-2493-43d2-954b-e2c4bd54243d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:38.416783 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b4ba1-2493-43d2-954b-e2c4bd54243d None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:38.418110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 665/1324] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:57:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:57:38.425558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7ca3b-b670-414c-b953-56d1b2c2fdb6 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:38.426173 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7ca3b-b670-414c-b953-56d1b2c2fdb6 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:38.426623 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7ca3b-b670-414c-b953-56d1b2c2fdb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:38.506052 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-8cc7ca3b-b670-414c-b953-56d1b2c2fdb6 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:38.507275 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-8cc7ca3b-b670-414c-b953-56d1b2c2fdb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:38.655832 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 660/1325] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:57:38.709098 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d9727-234d-4e45-ae2a-828a5bf3b7f3 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:38.709381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d9727-234d-4e45-ae2a-828a5bf3b7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:38.709660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4d9727-234d-4e45-ae2a-828a5bf3b7f3 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:38.711161 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 666/1326] 10.222.0.46 () {54 vars in 965 bytes} [Tue Aug 2 08:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:57:38.717993 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a63e33-8793-4ef1-9369-b41b2d054933 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:38.718290 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a63e33-8793-4ef1-9369-b41b2d054933 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:38.718573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a63e33-8793-4ef1-9369-b41b2d054933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:38.884240 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:38.948639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-e2a63e33-8793-4ef1-9369-b41b2d054933 None None] MFA Rules not processed for user `ddf6f4d84bb34d2f810a0f6cec36da4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:38.950152 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e2a63e33-8793-4ef1-9369-b41b2d054933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:39.050333 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 661/1327] 10.222.0.46 () {58 vars in 1062 bytes} [Tue Aug 2 08:57:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:57:39.060126 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d9727-234d-4e45-ae2a-828a5bf3b7f3 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:39.061650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ac4d9727-234d-4e45-ae2a-828a5bf3b7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:39.213565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4d9727-234d-4e45-ae2a-828a5bf3b7f3 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:39.309629 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83cdb06-1fe1-4585-b680-3b435590f5e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:39.310366 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cdb06-1fe1-4585-b680-3b435590f5e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:39.310874 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cdb06-1fe1-4585-b680-3b435590f5e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:39.311193 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83cdb06-1fe1-4585-b680-3b435590f5e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:39.313031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f83cdb06-1fe1-4585-b680-3b435590f5e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:39.338123 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:39.483202 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83cdb06-1fe1-4585-b680-3b435590f5e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:39.484663 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83cdb06-1fe1-4585-b680-3b435590f5e0 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:39.505629 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 667/1328] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:57:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:57:42.057928 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10bc7f-c447-454e-8371-0f26078ac40a None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:42.058526 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10bc7f-c447-454e-8371-0f26078ac40a None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:42.058526 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea10bc7f-c447-454e-8371-0f26078ac40a None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:42.059923 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 662/1329] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:57:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:57:42.065879 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abaf44f-4637-4fae-b28a-192246e85cd6 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:42.066476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abaf44f-4637-4fae-b28a-192246e85cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:42.066665 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abaf44f-4637-4fae-b28a-192246e85cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:42.160330 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-7abaf44f-4637-4fae-b28a-192246e85cd6 None None] MFA Rules not processed for user `7e1257a546984dca904f536574103621`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:42.162338 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7abaf44f-4637-4fae-b28a-192246e85cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:42.249412 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:42.321456 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 668/1330] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:57:42 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:57:42.362606 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10bc7f-c447-454e-8371-0f26078ac40a None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:42.364040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-ea10bc7f-c447-454e-8371-0f26078ac40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:42.499819 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea10bc7f-c447-454e-8371-0f26078ac40a None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:42.529313 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:42.612734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a2c53d-4e7a-4db5-95fe-69b2c5bb98db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:42.613287 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2c53d-4e7a-4db5-95fe-69b2c5bb98db service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:42.613573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2c53d-4e7a-4db5-95fe-69b2c5bb98db service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:42.613849 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a2c53d-4e7a-4db5-95fe-69b2c5bb98db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:42.615859 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-96a2c53d-4e7a-4db5-95fe-69b2c5bb98db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:42.797564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2c53d-4e7a-4db5-95fe-69b2c5bb98db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:42.799225 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a2c53d-4e7a-4db5-95fe-69b2c5bb98db service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:42.824665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 663/1331] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:57:42 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 08:57:45.969658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546c00b-967b-46b6-a341-d7bfbb8c4100 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:45.970234 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546c00b-967b-46b6-a341-d7bfbb8c4100 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:45.970428 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546c00b-967b-46b6-a341-d7bfbb8c4100 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:45.972666 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 669/1332] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:57:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:57:45.979111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b9b49-ca29-4fa0-867a-d70711fe601c None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:45.979815 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b9b49-ca29-4fa0-867a-d70711fe601c None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:45.979815 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b9b49-ca29-4fa0-867a-d70711fe601c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:46.063134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:46.066999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-4b2b9b49-ca29-4fa0-867a-d70711fe601c None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:46.068288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4b2b9b49-ca29-4fa0-867a-d70711fe601c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:46.185531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 664/1333] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:57:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:57:46.227416 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546c00b-967b-46b6-a341-d7bfbb8c4100 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:46.229761 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8546c00b-967b-46b6-a341-d7bfbb8c4100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:46.351673 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546c00b-967b-46b6-a341-d7bfbb8c4100 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:46.464385 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde1f6d-c71d-487e-a222-25ceb77a9905 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:46.465392 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde1f6d-c71d-487e-a222-25ceb77a9905 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:46.465767 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde1f6d-c71d-487e-a222-25ceb77a9905 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:46.466267 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde1f6d-c71d-487e-a222-25ceb77a9905 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:46.468264 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bdde1f6d-c71d-487e-a222-25ceb77a9905 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:46.504922 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:46.615590 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde1f6d-c71d-487e-a222-25ceb77a9905 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:46.619029 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde1f6d-c71d-487e-a222-25ceb77a9905 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:46.653925 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 670/1334] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:57:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:57:48.830708 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e193be6-5cac-4086-9ed1-16fd69044ed0 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:48.831273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e193be6-5cac-4086-9ed1-16fd69044ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:48.831273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e193be6-5cac-4086-9ed1-16fd69044ed0 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:48.832650 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 665/1335] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:57:48.838312 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc066e13-6ac0-44b0-8a71-70d60f992d83 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:48.838812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc066e13-6ac0-44b0-8a71-70d60f992d83 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:48.838926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc066e13-6ac0-44b0-8a71-70d60f992d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:48.918676 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-dc066e13-6ac0-44b0-8a71-70d60f992d83 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:48.919886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-dc066e13-6ac0-44b0-8a71-70d60f992d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:48.998943 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:49.023162 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 671/1336] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:57:49.064691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e193be6-5cac-4086-9ed1-16fd69044ed0 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:49.066320 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-9e193be6-5cac-4086-9ed1-16fd69044ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:49.175648 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e193be6-5cac-4086-9ed1-16fd69044ed0 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:49.206339 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:49.262701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ba760-36fd-415c-9873-5204f29b6707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:49.263384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ba760-36fd-415c-9873-5204f29b6707 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:49.263984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ba760-36fd-415c-9873-5204f29b6707 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:49.264266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ba760-36fd-415c-9873-5204f29b6707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:49.265991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-6b0ba760-36fd-415c-9873-5204f29b6707 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:49.390569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ba760-36fd-415c-9873-5204f29b6707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:49.392016 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ba760-36fd-415c-9873-5204f29b6707 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:49.419729 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 666/1337] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:57:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:57:52.390341 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fa610-b44d-4c1a-a10f-ed8c3222b7da None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:52.390863 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fa610-b44d-4c1a-a10f-ed8c3222b7da None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:52.390863 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fa610-b44d-4c1a-a10f-ed8c3222b7da None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:52.392280 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 672/1338] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:57:52.397935 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e149a6-dea6-4fd4-8937-13ae03c48aeb None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:52.398666 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e149a6-dea6-4fd4-8937-13ae03c48aeb None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:52.398666 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e149a6-dea6-4fd4-8937-13ae03c48aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:52.478434 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-c5e149a6-dea6-4fd4-8937-13ae03c48aeb None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:52.479659 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-c5e149a6-dea6-4fd4-8937-13ae03c48aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:52.488438 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:52.580560 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 667/1339] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:57:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:57:52.617499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fa610-b44d-4c1a-a10f-ed8c3222b7da None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:52.618615 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-364fa610-b44d-4c1a-a10f-ed8c3222b7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:52.733314 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fa610-b44d-4c1a-a10f-ed8c3222b7da None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:52.813990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b0840-186b-4b35-bc98-6d92c04d347a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:52.814606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b0840-186b-4b35-bc98-6d92c04d347a service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:52.814891 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b0840-186b-4b35-bc98-6d92c04d347a service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:52.815168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b0840-186b-4b35-bc98-6d92c04d347a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:52.816945 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-510b0840-186b-4b35-bc98-6d92c04d347a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:52.855179 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:52.935876 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b0840-186b-4b35-bc98-6d92c04d347a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:52.940328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b0840-186b-4b35-bc98-6d92c04d347a service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:52.957959 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 673/1340] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:57:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:57:57.296857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2314381-d7ae-4482-8fbb-d6136d729105 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:57.297402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2314381-d7ae-4482-8fbb-d6136d729105 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:57.297402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2314381-d7ae-4482-8fbb-d6136d729105 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:57.299056 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 668/1341] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:57:57.304372 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0781a-d52f-4340-9b5f-f2d507cf63a8 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:57.304852 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0781a-d52f-4340-9b5f-f2d507cf63a8 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:57.304970 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e0781a-d52f-4340-9b5f-f2d507cf63a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:57.384888 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-60e0781a-d52f-4340-9b5f-f2d507cf63a8 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:57:57.386159 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-60e0781a-d52f-4340-9b5f-f2d507cf63a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:57.485573 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:57.505213 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 674/1342] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:57:57.537850 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2314381-d7ae-4482-8fbb-d6136d729105 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:57:57.540652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e2314381-d7ae-4482-8fbb-d6136d729105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:57.668172 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2314381-d7ae-4482-8fbb-d6136d729105 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:57:57.708718 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:57:57.746546 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff70bb81-e0b5-4352-8c9e-8dc193d5d752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:57:57.747167 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70bb81-e0b5-4352-8c9e-8dc193d5d752 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:57:57.747450 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70bb81-e0b5-4352-8c9e-8dc193d5d752 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:57:57.747727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70bb81-e0b5-4352-8c9e-8dc193d5d752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:57:57.749344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-ff70bb81-e0b5-4352-8c9e-8dc193d5d752 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:57:57.868603 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff70bb81-e0b5-4352-8c9e-8dc193d5d752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:57:57.870003 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff70bb81-e0b5-4352-8c9e-8dc193d5d752 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:57:57.890507 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 669/1343] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:57:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:58:01.280760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f58f5b-0f46-46d4-9afd-bc979962cb65 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:01.281333 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f58f5b-0f46-46d4-9afd-bc979962cb65 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:01.281333 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f58f5b-0f46-46d4-9afd-bc979962cb65 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:01.282621 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 675/1344] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:58:01.288124 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b76bcc-bbd7-4e03-a84e-a8ddc09aa04c None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:01.288721 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b76bcc-bbd7-4e03-a84e-a8ddc09aa04c None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:01.288721 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b76bcc-bbd7-4e03-a84e-a8ddc09aa04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:01.364930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-94b76bcc-bbd7-4e03-a84e-a8ddc09aa04c None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:01.366207 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-94b76bcc-bbd7-4e03-a84e-a8ddc09aa04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:01.385867 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:01.499252 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 670/1345] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:58:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:58:01.538891 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f58f5b-0f46-46d4-9afd-bc979962cb65 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:01.540852 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-76f58f5b-0f46-46d4-9afd-bc979962cb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:01.659809 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f58f5b-0f46-46d4-9afd-bc979962cb65 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:01.750829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e492c042-0f56-4be8-b4e0-3898a7822325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:01.751511 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492c042-0f56-4be8-b4e0-3898a7822325 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:01.751936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492c042-0f56-4be8-b4e0-3898a7822325 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:01.752227 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e492c042-0f56-4be8-b4e0-3898a7822325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:01.753992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e492c042-0f56-4be8-b4e0-3898a7822325 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:01.804062 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:01.881413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492c042-0f56-4be8-b4e0-3898a7822325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:01.882915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e492c042-0f56-4be8-b4e0-3898a7822325 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:01.903841 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 676/1346] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:58:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 08:58:06.847653 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465d505-09fa-40df-886d-c9a6ef3786d2 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:06.848178 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465d505-09fa-40df-886d-c9a6ef3786d2 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:06.848277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465d505-09fa-40df-886d-c9a6ef3786d2 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:06.849775 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 671/1347] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:58:06.855967 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781bcc4-851b-4db0-aa51-ba3f7af2893b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:06.862802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781bcc4-851b-4db0-aa51-ba3f7af2893b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:06.862802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781bcc4-851b-4db0-aa51-ba3f7af2893b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:06.936996 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f781bcc4-851b-4db0-aa51-ba3f7af2893b None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:06.938843 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f781bcc4-851b-4db0-aa51-ba3f7af2893b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:07.048818 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:07.054178 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 677/1348] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 08:58:07.092600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465d505-09fa-40df-886d-c9a6ef3786d2 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:07.094562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-a465d505-09fa-40df-886d-c9a6ef3786d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:07.205606 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465d505-09fa-40df-886d-c9a6ef3786d2 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:07.258406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:07.288473 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca2d92-0ebe-4a27-8da8-50d0b664a2f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:07.289118 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca2d92-0ebe-4a27-8da8-50d0b664a2f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:07.289643 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca2d92-0ebe-4a27-8da8-50d0b664a2f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:07.289998 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca2d92-0ebe-4a27-8da8-50d0b664a2f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:07.292182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-66ca2d92-0ebe-4a27-8da8-50d0b664a2f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:07.405773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca2d92-0ebe-4a27-8da8-50d0b664a2f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:07.407257 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca2d92-0ebe-4a27-8da8-50d0b664a2f2 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:07.426234 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 672/1349] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:58:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:58:12.755079 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcbd33-049e-41a4-a65b-b9fd18dc06e7 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:12.755570 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcbd33-049e-41a4-a65b-b9fd18dc06e7 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:12.755655 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bcbd33-049e-41a4-a65b-b9fd18dc06e7 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:12.757667 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 678/1350] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 2 08:58:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:58:12.764326 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32c8ad-4faf-43ee-8ef5-33d689675977 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:12.764908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32c8ad-4faf-43ee-8ef5-33d689675977 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:12.765139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32c8ad-4faf-43ee-8ef5-33d689675977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:12.940006 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-fa32c8ad-4faf-43ee-8ef5-33d689675977 None None] MFA Rules not processed for user `cad5cc9927374836b0cc74db85b686fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:12.941954 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-fa32c8ad-4faf-43ee-8ef5-33d689675977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:12.990422 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:13.128891 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 673/1351] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 2 08:58:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:58:17.880703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e5401-7ee8-4ec2-afc0-fb885b555eba None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:17.881602 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e5401-7ee8-4ec2-afc0-fb885b555eba None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:17.882085 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e5401-7ee8-4ec2-afc0-fb885b555eba None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:17.883868 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 679/1352] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 2 08:58:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:58:17.889581 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6fb8e-dc6f-4d03-b081-2a314573776f None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:17.890433 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6fb8e-dc6f-4d03-b081-2a314573776f None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:17.890867 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e6fb8e-dc6f-4d03-b081-2a314573776f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:18.006208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-67e6fb8e-dc6f-4d03-b081-2a314573776f None None] MFA Rules not processed for user `cad5cc9927374836b0cc74db85b686fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:18.007534 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-67e6fb8e-dc6f-4d03-b081-2a314573776f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:18.127302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:18.183343 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 674/1353] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 2 08:58:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:58:20.832845 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30508959-65e4-4b0c-9ce3-f642810e2d6a None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:20.833779 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30508959-65e4-4b0c-9ce3-f642810e2d6a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:20.833779 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30508959-65e4-4b0c-9ce3-f642810e2d6a None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:20.835047 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 680/1354] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 2 08:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:58:20.841528 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263de64-a073-4ce5-bc94-df23c42861cd None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:20.842320 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263de64-a073-4ce5-bc94-df23c42861cd None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:20.842529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263de64-a073-4ce5-bc94-df23c42861cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:20.942925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-d263de64-a073-4ce5-bc94-df23c42861cd None None] MFA Rules not processed for user `cad5cc9927374836b0cc74db85b686fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:20.944484 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-d263de64-a073-4ce5-bc94-df23c42861cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:21.079169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 675/1355] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 2 08:58:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:58:24.408997 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4c14a-f239-4ad7-88c5-4075016219bb None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:24.409729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4c14a-f239-4ad7-88c5-4075016219bb None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:24.410043 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4c14a-f239-4ad7-88c5-4075016219bb None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:24.411637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 681/1356] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:58:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:58:24.417939 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f6f6a-4978-4f4d-b9b6-63ac771fd16d None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:24.418783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f6f6a-4978-4f4d-b9b6-63ac771fd16d None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:24.419059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5f6f6a-4978-4f4d-b9b6-63ac771fd16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:24.466444 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:24.518058 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-5c5f6f6a-4978-4f4d-b9b6-63ac771fd16d None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:24.519178 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-5c5f6f6a-4978-4f4d-b9b6-63ac771fd16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:24.701457 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 676/1357] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:58:24.779371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcfd50-bd00-43fc-8d1c-60d4617fb95e None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:24.780018 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcfd50-bd00-43fc-8d1c-60d4617fb95e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:24.780715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbcfd50-bd00-43fc-8d1c-60d4617fb95e None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:24.782406 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 682/1358] 10.222.0.46 () {54 vars in 962 bytes} [Tue Aug 2 08:58:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:58:24.805081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e0d51-b433-40b0-beaf-3706bb14155e None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:24.805721 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e0d51-b433-40b0-beaf-3706bb14155e None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:24.806208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e0d51-b433-40b0-beaf-3706bb14155e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:24.908208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-bf2e0d51-b433-40b0-beaf-3706bb14155e None None] MFA Rules not processed for user `3191d804e8dc4b1a85f490319833ec07`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:24.909835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-bf2e0d51-b433-40b0-beaf-3706bb14155e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:25.004381 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:25.089920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 677/1359] 10.222.0.46 () {58 vars in 1059 bytes} [Tue Aug 2 08:58:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:58:25.097944 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbcfd50-bd00-43fc-8d1c-60d4617fb95e None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:25.099449 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4dbcfd50-bd00-43fc-8d1c-60d4617fb95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:25.252880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbcfd50-bd00-43fc-8d1c-60d4617fb95e None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:25.332001 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:25.375516 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee96ca5-15ed-469c-b01c-18d04e20b83b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:25.376297 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee96ca5-15ed-469c-b01c-18d04e20b83b service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:25.376684 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee96ca5-15ed-469c-b01c-18d04e20b83b service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:25.377062 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee96ca5-15ed-469c-b01c-18d04e20b83b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:25.378851 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5ee96ca5-15ed-469c-b01c-18d04e20b83b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:25.475628 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee96ca5-15ed-469c-b01c-18d04e20b83b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:25.477390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee96ca5-15ed-469c-b01c-18d04e20b83b service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:25.497171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 683/1360] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:58:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:58:27.787280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece12bbf-dddb-432e-bced-dc7939f95c64 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:27.789007 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece12bbf-dddb-432e-bced-dc7939f95c64 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:27.790729 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece12bbf-dddb-432e-bced-dc7939f95c64 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:27.792692 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 678/1361] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:58:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:58:27.800853 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88894c11-7dd1-46dd-9286-ff8f4fe0d6f9 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:27.801663 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88894c11-7dd1-46dd-9286-ff8f4fe0d6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:27.801834 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88894c11-7dd1-46dd-9286-ff8f4fe0d6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:27.881502 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-88894c11-7dd1-46dd-9286-ff8f4fe0d6f9 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:27.882801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-88894c11-7dd1-46dd-9286-ff8f4fe0d6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:27.929525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:27.995804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 684/1362] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:58:28.046184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece12bbf-dddb-432e-bced-dc7939f95c64 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:28.047853 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-ece12bbf-dddb-432e-bced-dc7939f95c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:28.292858 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece12bbf-dddb-432e-bced-dc7939f95c64 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:28.306856 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:28.479185 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb47598e-0f6c-447c-9b0a-661f8466f8fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:28.479185 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47598e-0f6c-447c-9b0a-661f8466f8fc service nova] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:28.479185 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47598e-0f6c-447c-9b0a-661f8466f8fc service nova] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:28.479185 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb47598e-0f6c-447c-9b0a-661f8466f8fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:28.482604 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-bb47598e-0f6c-447c-9b0a-661f8466f8fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:28.627654 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb47598e-0f6c-447c-9b0a-661f8466f8fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:28.630269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb47598e-0f6c-447c-9b0a-661f8466f8fc service nova] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:28.664124 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 679/1363] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:58:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:58:31.296836 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcc313-c8af-431c-a84c-567c65f71022 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:31.297766 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcc313-c8af-431c-a84c-567c65f71022 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:31.298271 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbcc313-c8af-431c-a84c-567c65f71022 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:31.299949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 685/1364] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:58:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:58:31.305672 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5f0cb-80dc-4319-b05e-550b87f977f4 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:31.306298 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5f0cb-80dc-4319-b05e-550b87f977f4 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:31.306298 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5f0cb-80dc-4319-b05e-550b87f977f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:31.326094 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:31.412165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-74d5f0cb-80dc-4319-b05e-550b87f977f4 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:31.412676 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-74d5f0cb-80dc-4319-b05e-550b87f977f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:31.595863 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 680/1365] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 08:58:31.648654 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad6033-65c8-4e60-890e-3a20325022cb None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:31.649219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad6033-65c8-4e60-890e-3a20325022cb None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:31.649502 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad6033-65c8-4e60-890e-3a20325022cb None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:31.651283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 686/1366] 10.222.0.46 () {54 vars in 962 bytes} [Tue Aug 2 08:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:58:31.659159 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703ed0b-10f5-40d6-ac5d-a1540efc72da None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:31.663221 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703ed0b-10f5-40d6-ac5d-a1540efc72da None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:31.663753 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703ed0b-10f5-40d6-ac5d-a1540efc72da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:31.808020 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-4703ed0b-10f5-40d6-ac5d-a1540efc72da None None] MFA Rules not processed for user `3191d804e8dc4b1a85f490319833ec07`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:31.810661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4703ed0b-10f5-40d6-ac5d-a1540efc72da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:31.864748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:31.985375 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 681/1367] 10.222.0.46 () {58 vars in 1059 bytes} [Tue Aug 2 08:58:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 08:58:31.993460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad6033-65c8-4e60-890e-3a20325022cb None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:31.998513 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-fbad6033-65c8-4e60-890e-3a20325022cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:32.184626 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad6033-65c8-4e60-890e-3a20325022cb None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:32.220741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:32.343395 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef16cc-44b4-4107-89a7-b782fd7e354e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:32.344485 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef16cc-44b4-4107-89a7-b782fd7e354e service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:32.344921 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef16cc-44b4-4107-89a7-b782fd7e354e service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:32.345298 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef16cc-44b4-4107-89a7-b782fd7e354e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:32.347432 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-fdef16cc-44b4-4107-89a7-b782fd7e354e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:32.467501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef16cc-44b4-4107-89a7-b782fd7e354e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:32.469245 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef16cc-44b4-4107-89a7-b782fd7e354e service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:32.489498 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 687/1368] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:58:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:58:33.786292 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852425e-eb3a-4cb9-8e8b-4f9b4f18c21b None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:33.786914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852425e-eb3a-4cb9-8e8b-4f9b4f18c21b None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:33.786914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852425e-eb3a-4cb9-8e8b-4f9b4f18c21b None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:33.789927 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 682/1369] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:58:33.794563 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c57ce9-26ac-45fd-8aa8-387765bba9a4 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:33.795080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c57ce9-26ac-45fd-8aa8-387765bba9a4 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:33.795080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c57ce9-26ac-45fd-8aa8-387765bba9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:33.829268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:33.880384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-72c57ce9-26ac-45fd-8aa8-387765bba9a4 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:33.881873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-72c57ce9-26ac-45fd-8aa8-387765bba9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:34.032423 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 688/1370] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:58:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:58:34.400114 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f65c1-b7c4-4f52-ae12-1b1daab6976a None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:34.401239 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f65c1-b7c4-4f52-ae12-1b1daab6976a None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:34.401831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f65c1-b7c4-4f52-ae12-1b1daab6976a None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:34.405010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 683/1371] 10.222.0.46 () {54 vars in 964 bytes} [Tue Aug 2 08:58:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:58:34.413304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c76cc-8fc2-4a50-a3f2-3a602fb9db1b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:34.413837 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c76cc-8fc2-4a50-a3f2-3a602fb9db1b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:34.414336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c76cc-8fc2-4a50-a3f2-3a602fb9db1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:34.521321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-132c76cc-8fc2-4a50-a3f2-3a602fb9db1b None None] MFA Rules not processed for user `6e39392b79bc439780415db7c8206098`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:34.522595 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-132c76cc-8fc2-4a50-a3f2-3a602fb9db1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:34.580931 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:34.622691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 689/1372] 10.222.0.46 () {58 vars in 1061 bytes} [Tue Aug 2 08:58:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:58:34.635113 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f65c1-b7c4-4f52-ae12-1b1daab6976a None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:34.635113 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-7b1f65c1-b7c4-4f52-ae12-1b1daab6976a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:34.779723 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f65c1-b7c4-4f52-ae12-1b1daab6976a None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:34.848781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:34.904059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5901cec-2a38-467b-8e17-2b7d96366759 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:34.905029 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5901cec-2a38-467b-8e17-2b7d96366759 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:34.905029 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5901cec-2a38-467b-8e17-2b7d96366759 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:34.905322 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5901cec-2a38-467b-8e17-2b7d96366759 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:34.907107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-a5901cec-2a38-467b-8e17-2b7d96366759 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:35.070699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5901cec-2a38-467b-8e17-2b7d96366759 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:35.072500 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5901cec-2a38-467b-8e17-2b7d96366759 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:35.096720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 684/1373] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:58:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:58:35.153672 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6125f3-e8ae-4d03-ba73-3a5f709f74cc None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:35.154572 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6125f3-e8ae-4d03-ba73-3a5f709f74cc None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:35.154805 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6125f3-e8ae-4d03-ba73-3a5f709f74cc None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:35.156345 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 690/1374] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:58:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:58:35.163385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3691ff-fef4-4657-8a77-11e1bda0882a None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:35.163385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3691ff-fef4-4657-8a77-11e1bda0882a None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:35.163887 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3691ff-fef4-4657-8a77-11e1bda0882a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:35.285735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-4c3691ff-fef4-4657-8a77-11e1bda0882a None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:35.288051 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4c3691ff-fef4-4657-8a77-11e1bda0882a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:35.333242 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:35.449134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 685/1375] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:58:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 08:58:35.510728 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6125f3-e8ae-4d03-ba73-3a5f709f74cc None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:35.514670 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2a6125f3-e8ae-4d03-ba73-3a5f709f74cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:35.713648 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6125f3-e8ae-4d03-ba73-3a5f709f74cc None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:35.776643 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:35.845715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7d789-a6e1-4fe0-9333-37e99dbf853c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:35.846328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7d789-a6e1-4fe0-9333-37e99dbf853c service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:35.846610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7d789-a6e1-4fe0-9333-37e99dbf853c service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:35.846888 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7d789-a6e1-4fe0-9333-37e99dbf853c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:35.848650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d2e7d789-a6e1-4fe0-9333-37e99dbf853c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:36.005144 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7d789-a6e1-4fe0-9333-37e99dbf853c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:36.006729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7d789-a6e1-4fe0-9333-37e99dbf853c service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:36.035831 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 691/1376] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:58:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:58:38.303847 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4bd34-a49f-4d0c-a602-fb855cabb1fe None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:38.304391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4bd34-a49f-4d0c-a602-fb855cabb1fe None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:38.304391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4bd34-a49f-4d0c-a602-fb855cabb1fe None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:38.305683 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 686/1377] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:58:38.312704 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761fbe2-2e0f-40f9-9ab4-f1f5e7356fad None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:38.313155 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761fbe2-2e0f-40f9-9ab4-f1f5e7356fad None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:38.313551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4761fbe2-2e0f-40f9-9ab4-f1f5e7356fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:38.396337 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-4761fbe2-2e0f-40f9-9ab4-f1f5e7356fad None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:38.397603 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4761fbe2-2e0f-40f9-9ab4-f1f5e7356fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:38.406566 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:38.533111 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 692/1378] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:58:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:58:38.593780 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4bd34-a49f-4d0c-a602-fb855cabb1fe None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:38.597095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f9a4bd34-a49f-4d0c-a602-fb855cabb1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:38.819958 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4bd34-a49f-4d0c-a602-fb855cabb1fe None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:38.824044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:38.959965 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2efce2f-ac2a-42c6-87c1-5cfccfbacdbb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:38.960609 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efce2f-ac2a-42c6-87c1-5cfccfbacdbb service nova] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:38.960892 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efce2f-ac2a-42c6-87c1-5cfccfbacdbb service nova] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:38.961167 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2efce2f-ac2a-42c6-87c1-5cfccfbacdbb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:38.964207 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f2efce2f-ac2a-42c6-87c1-5cfccfbacdbb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:39.153224 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2efce2f-ac2a-42c6-87c1-5cfccfbacdbb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:39.155819 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2efce2f-ac2a-42c6-87c1-5cfccfbacdbb service nova] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:39.195393 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 687/1379] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:58:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:58:41.602276 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72072b-1a23-4c64-a57e-e8c346879ea3 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:41.602810 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72072b-1a23-4c64-a57e-e8c346879ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:41.602927 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72072b-1a23-4c64-a57e-e8c346879ea3 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:41.605275 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 693/1380] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:58:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:58:41.611376 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa7b1f-cbbc-4431-b228-ebccdb73bf49 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:41.611784 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa7b1f-cbbc-4431-b228-ebccdb73bf49 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:41.611889 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa7b1f-cbbc-4431-b228-ebccdb73bf49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:41.627369 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:41.693292 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-12fa7b1f-cbbc-4431-b228-ebccdb73bf49 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:41.694557 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-12fa7b1f-cbbc-4431-b228-ebccdb73bf49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:41.862903 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 688/1381] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:58:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:58:41.923237 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72072b-1a23-4c64-a57e-e8c346879ea3 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:41.927204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1a72072b-1a23-4c64-a57e-e8c346879ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:42.124071 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:42.132793 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72072b-1a23-4c64-a57e-e8c346879ea3 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:42.289383 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73865b47-df4c-48ad-8de5-d91bf1c0609e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:42.290111 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73865b47-df4c-48ad-8de5-d91bf1c0609e service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:42.290389 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73865b47-df4c-48ad-8de5-d91bf1c0609e service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:42.290824 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73865b47-df4c-48ad-8de5-d91bf1c0609e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:42.293191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-73865b47-df4c-48ad-8de5-d91bf1c0609e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:42.412842 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73865b47-df4c-48ad-8de5-d91bf1c0609e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:42.415461 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73865b47-df4c-48ad-8de5-d91bf1c0609e service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:42.428048 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:42.441038 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 694/1382] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:58:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:58:44.911822 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1517b87-2cd2-409e-bf72-13c9195c0a1d None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:44.912618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1517b87-2cd2-409e-bf72-13c9195c0a1d None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:44.912618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1517b87-2cd2-409e-bf72-13c9195c0a1d None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:44.914504 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 689/1383] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:58:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:58:44.920768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558810ef-1984-486f-9479-8ab3bf47a5ad None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:44.921353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558810ef-1984-486f-9479-8ab3bf47a5ad None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:44.921536 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558810ef-1984-486f-9479-8ab3bf47a5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:45.005643 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-558810ef-1984-486f-9479-8ab3bf47a5ad None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:45.007647 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-558810ef-1984-486f-9479-8ab3bf47a5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:45.123248 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 695/1384] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:58:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:58:45.174256 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1517b87-2cd2-409e-bf72-13c9195c0a1d None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:45.175861 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-b1517b87-2cd2-409e-bf72-13c9195c0a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:45.272472 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:45.347730 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1517b87-2cd2-409e-bf72-13c9195c0a1d None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:45.486045 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584801e-d2aa-471b-8a5e-e9059baade0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:45.487182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584801e-d2aa-471b-8a5e-e9059baade0a service nova] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:45.487702 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584801e-d2aa-471b-8a5e-e9059baade0a service nova] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:45.488212 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584801e-d2aa-471b-8a5e-e9059baade0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:45.491493 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-0584801e-d2aa-471b-8a5e-e9059baade0a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:45.549323 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:45.594328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0584801e-d2aa-471b-8a5e-e9059baade0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:45.605874 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0584801e-d2aa-471b-8a5e-e9059baade0a service nova] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:45.618320 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 690/1385] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:58:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:58:47.787693 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458a9c8-15eb-415a-b58e-baa540774dd3 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:47.788405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458a9c8-15eb-415a-b58e-baa540774dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:47.788726 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458a9c8-15eb-415a-b58e-baa540774dd3 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:47.791387 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 696/1386] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:58:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:58:47.801020 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154e515-8e41-4f4e-91ba-a6f4cc7de67d None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:47.801675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154e515-8e41-4f4e-91ba-a6f4cc7de67d None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:47.801675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154e515-8e41-4f4e-91ba-a6f4cc7de67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:47.922334 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-f154e515-8e41-4f4e-91ba-a6f4cc7de67d None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:47.924363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f154e515-8e41-4f4e-91ba-a6f4cc7de67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:48.060126 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 691/1387] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:58:48.109534 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458a9c8-15eb-415a-b58e-baa540774dd3 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:48.111436 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e458a9c8-15eb-415a-b58e-baa540774dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:48.217871 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:48.351854 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458a9c8-15eb-415a-b58e-baa540774dd3 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:48.528607 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5df943-cffd-45aa-872b-5538b402b583 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:48.529657 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5df943-cffd-45aa-872b-5538b402b583 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:48.530190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5df943-cffd-45aa-872b-5538b402b583 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:48.530604 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5df943-cffd-45aa-872b-5538b402b583 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:48.533629 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-fb5df943-cffd-45aa-872b-5538b402b583 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:48.594735 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:48.659030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5df943-cffd-45aa-872b-5538b402b583 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:48.660808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5df943-cffd-45aa-872b-5538b402b583 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:48.680005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 697/1388] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:58:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:58:50.973413 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43891c1e-6e90-4e1c-be42-ff8f2db27ca4 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:50.974048 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43891c1e-6e90-4e1c-be42-ff8f2db27ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:50.974048 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43891c1e-6e90-4e1c-be42-ff8f2db27ca4 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:50.975421 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 692/1389] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:58:50.981635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322989ce-acdc-4c04-8673-502b0e50cc84 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:50.982200 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322989ce-acdc-4c04-8673-502b0e50cc84 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:50.982368 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322989ce-acdc-4c04-8673-502b0e50cc84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:51.058048 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-322989ce-acdc-4c04-8673-502b0e50cc84 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:51.059282 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-322989ce-acdc-4c04-8673-502b0e50cc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:51.185992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 698/1390] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:58:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:58:51.236873 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43891c1e-6e90-4e1c-be42-ff8f2db27ca4 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:51.238438 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-43891c1e-6e90-4e1c-be42-ff8f2db27ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:51.272905 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:51.412315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43891c1e-6e90-4e1c-be42-ff8f2db27ca4 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:51.425424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d0ca4-2a2a-449f-8971-6c8233dab35b None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:51.432118 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d0ca4-2a2a-449f-8971-6c8233dab35b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:51.432118 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d0ca4-2a2a-449f-8971-6c8233dab35b None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:51.432118 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 699/1391] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:58:51.433639 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44cf49-ae11-4938-8295-a62c76142759 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:51.434034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44cf49-ae11-4938-8295-a62c76142759 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:51.434357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44cf49-ae11-4938-8295-a62c76142759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:51.451442 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:51.517829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9d44cf49-ae11-4938-8295-a62c76142759 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:51.519073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9d44cf49-ae11-4938-8295-a62c76142759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:51.546878 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:51.564402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7aa2a4-dda5-4da9-967f-e99522ed873a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:51.565035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7aa2a4-dda5-4da9-967f-e99522ed873a service nova] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:51.565319 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7aa2a4-dda5-4da9-967f-e99522ed873a service nova] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:51.565595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7aa2a4-dda5-4da9-967f-e99522ed873a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:51.567306 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-9b7aa2a4-dda5-4da9-967f-e99522ed873a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:51.645114 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 700/1392] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:58:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 08:58:51.681102 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7aa2a4-dda5-4da9-967f-e99522ed873a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:51.682804 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7aa2a4-dda5-4da9-967f-e99522ed873a service nova] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:51.703940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5443e-e6e3-4f4a-af1c-21999224c91f None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:51.704264 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5443e-e6e3-4f4a-af1c-21999224c91f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:51.704463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab5443e-e6e3-4f4a-af1c-21999224c91f None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:51.706008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 701/1393] 10.222.0.46 () {54 vars in 966 bytes} [Tue Aug 2 08:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:58:51.706677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 693/1394] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:58:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:58:51.718990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ed4d4-06af-4dbe-b404-5df8edb4d171 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:51.719278 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ed4d4-06af-4dbe-b404-5df8edb4d171 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:51.719584 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ed4d4-06af-4dbe-b404-5df8edb4d171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:51.807391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-531ed4d4-06af-4dbe-b404-5df8edb4d171 None None] MFA Rules not processed for user `4abd1512ad224da896c6f2257f9602e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:51.808675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-531ed4d4-06af-4dbe-b404-5df8edb4d171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:51.849333 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:51.900666 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 702/1395] 10.222.0.46 () {58 vars in 1063 bytes} [Tue Aug 2 08:58:51 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:58:51.908877 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7aa2a4-dda5-4da9-967f-e99522ed873a service nova] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:51.910131 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-9b7aa2a4-dda5-4da9-967f-e99522ed873a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:52.013221 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7aa2a4-dda5-4da9-967f-e99522ed873a service nova] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:52.057570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:52.101038 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f01ad-4c51-4eb5-bfe1-42aada996a72 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4abd1512ad224da896c6f2257f9602e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:52.101686 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f01ad-4c51-4eb5-bfe1-42aada996a72 service glance] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:52.102177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f01ad-4c51-4eb5-bfe1-42aada996a72 service glance] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:52.102456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f01ad-4c51-4eb5-bfe1-42aada996a72 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:52.104273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-151f01ad-4c51-4eb5-bfe1-42aada996a72 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:52.228708 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f01ad-4c51-4eb5-bfe1-42aada996a72 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:52.230525 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f01ad-4c51-4eb5-bfe1-42aada996a72 service glance] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:52.251129 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 694/1396] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:58:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:58:53.365131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd9923-2579-49f1-919e-15f62e3d1f6d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:53.365645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd9923-2579-49f1-919e-15f62e3d1f6d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:53.366085 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd9923-2579-49f1-919e-15f62e3d1f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:53.443593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-a1fd9923-2579-49f1-919e-15f62e3d1f6d None None] MFA Rules not processed for user `ed8114b7a28741268fdff71c866dbc77`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:53.444814 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a1fd9923-2579-49f1-919e-15f62e3d1f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:53.555563 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 703/1397] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 2 08:58:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:58:53.604845 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1c1fe-432d-471b-a9fd-f4c4d62d70e7 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:53.604845 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1c1fe-432d-471b-a9fd-f4c4d62d70e7 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:53.605522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d1c1fe-432d-471b-a9fd-f4c4d62d70e7 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:53.606528 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 695/1398] 10.222.0.46 () {54 vars in 943 bytes} [Tue Aug 2 08:58:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:58:53.613784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee09212-e026-4d49-8096-3232a0692d75 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:53.614170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee09212-e026-4d49-8096-3232a0692d75 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:53.614543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee09212-e026-4d49-8096-3232a0692d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:53.683435 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:53.704331 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9ee09212-e026-4d49-8096-3232a0692d75 None None] MFA Rules not processed for user `ff92ef3b9da74ccab3fbc43f852e3ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:53.705702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9ee09212-e026-4d49-8096-3232a0692d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:53.823477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 704/1399] 10.222.0.46 () {58 vars in 1040 bytes} [Tue Aug 2 08:58:53 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:58:53.833245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1c1fe-432d-471b-a9fd-f4c4d62d70e7 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:53.835104 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-69d1c1fe-432d-471b-a9fd-f4c4d62d70e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:53.931309 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:54.000199 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d1c1fe-432d-471b-a9fd-f4c4d62d70e7 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:54.037844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed1dc0-47ed-4e79-b3b1-06b92ebb32d2 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:54.038347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed1dc0-47ed-4e79-b3b1-06b92ebb32d2 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:54.038807 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed1dc0-47ed-4e79-b3b1-06b92ebb32d2 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:54.041830 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 705/1400] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:58:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:58:54.047962 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82cac0-b2b6-45a5-8c90-9c4db31d9563 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:54.048198 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82cac0-b2b6-45a5-8c90-9c4db31d9563 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:54.048460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e82cac0-b2b6-45a5-8c90-9c4db31d9563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:54.123519 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-2e82cac0-b2b6-45a5-8c90-9c4db31d9563 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:54.124805 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2e82cac0-b2b6-45a5-8c90-9c4db31d9563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:54.128844 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7af93e-41c5-43e9-b3b0-74f4f2e86562 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff92ef3b9da74ccab3fbc43f852e3ad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:54.129479 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7af93e-41c5-43e9-b3b0-74f4f2e86562 service swift] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:54.129758 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7af93e-41c5-43e9-b3b0-74f4f2e86562 service swift] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:54.130063 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7af93e-41c5-43e9-b3b0-74f4f2e86562 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:54.131849 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-df7af93e-41c5-43e9-b3b0-74f4f2e86562 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:54.189486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:54.198858 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:54.230367 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7af93e-41c5-43e9-b3b0-74f4f2e86562 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:54.232023 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7af93e-41c5-43e9-b3b0-74f4f2e86562 service swift] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:54.241031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 696/1401] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 2 08:58:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 08:58:54.253516 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 706/1402] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 08:58:54.327026 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7af93e-41c5-43e9-b3b0-74f4f2e86562 service swift] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:54.327026 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-df7af93e-41c5-43e9-b3b0-74f4f2e86562 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:54.407889 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7af93e-41c5-43e9-b3b0-74f4f2e86562 service swift] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:54.535975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:54.547897 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a1edaf-ee4d-4108-9ecb-0ed138ef9779 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:54.548738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1edaf-ee4d-4108-9ecb-0ed138ef9779 service nova] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:54.549022 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1edaf-ee4d-4108-9ecb-0ed138ef9779 service nova] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:54.549301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a1edaf-ee4d-4108-9ecb-0ed138ef9779 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:54.551042 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-21a1edaf-ee4d-4108-9ecb-0ed138ef9779 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:54.664534 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1edaf-ee4d-4108-9ecb-0ed138ef9779 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:54.668560 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a1edaf-ee4d-4108-9ecb-0ed138ef9779 service nova] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:54.697785 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 697/1403] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:58:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:58:57.031745 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d12a4-7819-4b85-8ac8-c907b4402edd None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:57.032355 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d12a4-7819-4b85-8ac8-c907b4402edd None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:57.032355 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d12a4-7819-4b85-8ac8-c907b4402edd None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:57.033718 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 707/1404] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:58:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:58:57.040259 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c22c09-fe94-4a99-a722-1773c30f5a8e None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:57.040859 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c22c09-fe94-4a99-a722-1773c30f5a8e None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:57.040859 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c22c09-fe94-4a99-a722-1773c30f5a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:57.136913 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-99c22c09-fe94-4a99-a722-1773c30f5a8e None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:58:57.138184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-99c22c09-fe94-4a99-a722-1773c30f5a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:57.219972 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:57.280465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 698/1405] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:58:57.320188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d12a4-7819-4b85-8ac8-c907b4402edd None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:58:57.322372 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e01d12a4-7819-4b85-8ac8-c907b4402edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:57.537701 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d12a4-7819-4b85-8ac8-c907b4402edd None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:58:57.593084 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:58:57.709551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12435fcd-31ab-4b24-947f-5fba78cf5721 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:58:57.710402 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12435fcd-31ab-4b24-947f-5fba78cf5721 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:58:57.710799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12435fcd-31ab-4b24-947f-5fba78cf5721 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:58:57.711127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12435fcd-31ab-4b24-947f-5fba78cf5721 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:58:57.713423 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-12435fcd-31ab-4b24-947f-5fba78cf5721 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:58:57.862659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12435fcd-31ab-4b24-947f-5fba78cf5721 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:58:57.864241 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12435fcd-31ab-4b24-947f-5fba78cf5721 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:58:57.886837 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 708/1406] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:58:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:59:00.211563 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b33add-c37a-49e5-b845-c0ae444f82eb None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:00.212096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b33add-c37a-49e5-b845-c0ae444f82eb None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:00.212096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b33add-c37a-49e5-b845-c0ae444f82eb None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:00.213464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 699/1407] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 08:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:59:00.219397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66350826-9412-4e86-ae3f-a5965d5ed67b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:00.220288 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66350826-9412-4e86-ae3f-a5965d5ed67b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:00.220288 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66350826-9412-4e86-ae3f-a5965d5ed67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:00.342956 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:00.351206 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-66350826-9412-4e86-ae3f-a5965d5ed67b None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:00.352903 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-66350826-9412-4e86-ae3f-a5965d5ed67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:00.506042 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 709/1408] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 08:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:59:00.543051 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b33add-c37a-49e5-b845-c0ae444f82eb None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:00.544605 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-30b33add-c37a-49e5-b845-c0ae444f82eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:00.742406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b33add-c37a-49e5-b845-c0ae444f82eb None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:00.813961 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:00.944035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaacd6b2-730c-4ae1-95e0-6eb738b83e70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:00.945135 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacd6b2-730c-4ae1-95e0-6eb738b83e70 service nova] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:00.945680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacd6b2-730c-4ae1-95e0-6eb738b83e70 service nova] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:00.946292 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacd6b2-730c-4ae1-95e0-6eb738b83e70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:00.949274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-aaacd6b2-730c-4ae1-95e0-6eb738b83e70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:01.115971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaacd6b2-730c-4ae1-95e0-6eb738b83e70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:01.118609 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaacd6b2-730c-4ae1-95e0-6eb738b83e70 service nova] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:01.151804 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 700/1409] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:59:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 609 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:59:10.487891 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=44 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:11.150133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=77 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:19.659254 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516f96d-fea1-4e8e-80bb-afe56b30b8db None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:19.659831 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516f96d-fea1-4e8e-80bb-afe56b30b8db None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:19.659831 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516f96d-fea1-4e8e-80bb-afe56b30b8db None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:19.661142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 710/1410] 10.222.0.46 () {54 vars in 918 bytes} [Tue Aug 2 08:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:59:19.668100 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c621f7b-617e-4a8b-88e6-e02a9d7beee7 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:19.668627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c621f7b-617e-4a8b-88e6-e02a9d7beee7 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:19.669105 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c621f7b-617e-4a8b-88e6-e02a9d7beee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:19.759246 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-5c621f7b-617e-4a8b-88e6-e02a9d7beee7 None None] MFA Rules not processed for user `cad5cc9927374836b0cc74db85b686fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:19.760686 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-5c621f7b-617e-4a8b-88e6-e02a9d7beee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:19.885569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 701/1411] 10.222.0.46 () {58 vars in 1015 bytes} [Tue Aug 2 08:59:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:59:19.903451 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dda56a-bc94-4b57-8111-728ee7fab411 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:19.903819 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dda56a-bc94-4b57-8111-728ee7fab411 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:19.904218 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dda56a-bc94-4b57-8111-728ee7fab411 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:19.905705 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 711/1412] 10.222.0.46 () {54 vars in 918 bytes} [Tue Aug 2 08:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:59:19.910973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d2183-8595-47dd-9264-e0bc6f84f993 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:19.911264 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d2183-8595-47dd-9264-e0bc6f84f993 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:19.911580 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d2183-8595-47dd-9264-e0bc6f84f993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:20.008372 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:20.011451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-918d2183-8595-47dd-9264-e0bc6f84f993 None None] MFA Rules not processed for user `3191d804e8dc4b1a85f490319833ec07`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:20.012720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-918d2183-8595-47dd-9264-e0bc6f84f993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:20.136575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 702/1413] 10.222.0.46 () {58 vars in 1015 bytes} [Tue Aug 2 08:59:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:59:22.710574 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86568526-1873-4139-8b71-897d05e29a35 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:22.711334 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86568526-1873-4139-8b71-897d05e29a35 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:22.711334 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86568526-1873-4139-8b71-897d05e29a35 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:22.713018 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 08:59:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:59:22.719756 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476851e2-f394-4281-ae58-f28a5310ee27 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:22.724112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476851e2-f394-4281-ae58-f28a5310ee27 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:22.724112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476851e2-f394-4281-ae58-f28a5310ee27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:22.808891 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-476851e2-f394-4281-ae58-f28a5310ee27 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:22.810249 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-476851e2-f394-4281-ae58-f28a5310ee27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:22.882899 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:22.945392 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 08:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:59:22.992461 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86568526-1873-4139-8b71-897d05e29a35 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:22.994092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-86568526-1873-4139-8b71-897d05e29a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:23.186687 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86568526-1873-4139-8b71-897d05e29a35 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:23.281618 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:23.334278 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33c565b-efcd-4cb2-8d75-f4867cd4318e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:23.335244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c565b-efcd-4cb2-8d75-f4867cd4318e service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:23.335621 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c565b-efcd-4cb2-8d75-f4867cd4318e service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:23.336245 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c565b-efcd-4cb2-8d75-f4867cd4318e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:23.338542 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b33c565b-efcd-4cb2-8d75-f4867cd4318e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:23.461531 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c565b-efcd-4cb2-8d75-f4867cd4318e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:23.463856 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33c565b-efcd-4cb2-8d75-f4867cd4318e service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:23.494484 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 713/1416] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:59:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:59:24.501724 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e4d5b-2b1f-435f-b211-a4622905ade3 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:24.502303 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e4d5b-2b1f-435f-b211-a4622905ade3 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:24.502303 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695e4d5b-2b1f-435f-b211-a4622905ade3 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:24.503609 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 08:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:59:24.509414 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6eb752-a566-4584-8145-135c62343e9e None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:24.509874 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6eb752-a566-4584-8145-135c62343e9e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:24.510043 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6eb752-a566-4584-8145-135c62343e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:24.612259 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-4a6eb752-a566-4584-8145-135c62343e9e None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:24.613525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4a6eb752-a566-4584-8145-135c62343e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:24.646507 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:24.738732 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 08:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 08:59:24.900500 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e4d5b-2b1f-435f-b211-a4622905ade3 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:24.902850 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-695e4d5b-2b1f-435f-b211-a4622905ade3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:25.095542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695e4d5b-2b1f-435f-b211-a4622905ade3 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:25.145617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:25.237160 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8cbc6-b802-4b30-a645-28b41e96c2b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:25.237778 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8cbc6-b802-4b30-a645-28b41e96c2b9 service nova] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:25.238081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8cbc6-b802-4b30-a645-28b41e96c2b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:25.238352 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8cbc6-b802-4b30-a645-28b41e96c2b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:25.240038 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-6ff8cbc6-b802-4b30-a645-28b41e96c2b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:25.351090 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8cbc6-b802-4b30-a645-28b41e96c2b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:25.353581 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8cbc6-b802-4b30-a645-28b41e96c2b9 service nova] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:25.380360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 705/1419] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:59:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:59:26.357749 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236c33a-7515-4a25-a0be-28f3130bca1b None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:26.358345 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236c33a-7515-4a25-a0be-28f3130bca1b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:26.358732 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236c33a-7515-4a25-a0be-28f3130bca1b None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:26.361329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 08:59:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:59:26.366874 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37730a7-665c-4377-81af-e4a3c54feb48 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:26.367522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37730a7-665c-4377-81af-e4a3c54feb48 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:26.367522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37730a7-665c-4377-81af-e4a3c54feb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:26.436136 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:26.464045 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-f37730a7-665c-4377-81af-e4a3c54feb48 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:26.466257 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f37730a7-665c-4377-81af-e4a3c54feb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:26.634694 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 08:59:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 08:59:26.802098 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236c33a-7515-4a25-a0be-28f3130bca1b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:26.803897 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0236c33a-7515-4a25-a0be-28f3130bca1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:26.995553 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236c33a-7515-4a25-a0be-28f3130bca1b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:27.009207 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:27.152639 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb91dd9-0c9d-4449-a286-e48ea105b890 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:27.153673 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb91dd9-0c9d-4449-a286-e48ea105b890 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:27.154256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb91dd9-0c9d-4449-a286-e48ea105b890 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:27.154812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb91dd9-0c9d-4449-a286-e48ea105b890 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:27.158241 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6fb91dd9-0c9d-4449-a286-e48ea105b890 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:27.295594 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb91dd9-0c9d-4449-a286-e48ea105b890 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:27.297108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb91dd9-0c9d-4449-a286-e48ea105b890 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:27.319358 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 716/1422] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:59:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:59:28.277930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979f9ee-af41-4416-83ff-1358f541f1ae None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:28.278537 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979f9ee-af41-4416-83ff-1358f541f1ae None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:28.278537 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979f9ee-af41-4416-83ff-1358f541f1ae None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:28.279869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 08:59:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:59:28.286540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9469ac-4277-4ab1-9514-6fd4d2a1ef20 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:28.287413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9469ac-4277-4ab1-9514-6fd4d2a1ef20 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:28.287604 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9469ac-4277-4ab1-9514-6fd4d2a1ef20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:28.315463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:28.388403 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-fd9469ac-4277-4ab1-9514-6fd4d2a1ef20 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:28.389646 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-fd9469ac-4277-4ab1-9514-6fd4d2a1ef20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:28.506329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 08:59:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:59:28.668415 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979f9ee-af41-4416-83ff-1358f541f1ae None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:28.670015 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-1979f9ee-af41-4416-83ff-1358f541f1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:28.807499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:28.849346 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979f9ee-af41-4416-83ff-1358f541f1ae None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:29.004664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7192ea03-91fb-4d4b-aa96-ce5d261c28eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:29.005739 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192ea03-91fb-4d4b-aa96-ce5d261c28eb service nova] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:29.006061 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192ea03-91fb-4d4b-aa96-ce5d261c28eb service nova] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:29.006397 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7192ea03-91fb-4d4b-aa96-ce5d261c28eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:29.009415 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-7192ea03-91fb-4d4b-aa96-ce5d261c28eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:29.106305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:29.121572 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7192ea03-91fb-4d4b-aa96-ce5d261c28eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:29.123334 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7192ea03-91fb-4d4b-aa96-ce5d261c28eb service nova] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:29.144352 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 708/1425] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:59:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 08:59:30.095676 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a028c38-6f44-43d3-b39c-24ba3e8ca8b0 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:30.096675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a028c38-6f44-43d3-b39c-24ba3e8ca8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:30.096675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a028c38-6f44-43d3-b39c-24ba3e8ca8b0 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:30.098455 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 08:59:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:59:30.110866 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d92a4b-33dd-4584-8022-6795cb21ae38 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:30.111434 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d92a4b-33dd-4584-8022-6795cb21ae38 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:30.111797 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d92a4b-33dd-4584-8022-6795cb21ae38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:30.192284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-52d92a4b-33dd-4584-8022-6795cb21ae38 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:30.193450 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-52d92a4b-33dd-4584-8022-6795cb21ae38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:30.321236 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 08:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:59:30.478616 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a028c38-6f44-43d3-b39c-24ba3e8ca8b0 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:30.480171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7a028c38-6f44-43d3-b39c-24ba3e8ca8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:30.573819 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:30.644848 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a028c38-6f44-43d3-b39c-24ba3e8ca8b0 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:30.789216 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4662268-e5a7-440d-b622-98288dc2443a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:30.790312 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4662268-e5a7-440d-b622-98288dc2443a service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:30.790589 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4662268-e5a7-440d-b622-98288dc2443a service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:30.790861 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4662268-e5a7-440d-b622-98288dc2443a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:30.792487 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b4662268-e5a7-440d-b622-98288dc2443a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:30.854498 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:30.916505 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4662268-e5a7-440d-b622-98288dc2443a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:30.918158 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4662268-e5a7-440d-b622-98288dc2443a service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:30.941178 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 719/1428] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:59:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:59:31.857798 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852bd936-08e3-4565-ae5b-7324e4c01ef0 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:31.858400 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852bd936-08e3-4565-ae5b-7324e4c01ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:31.858400 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852bd936-08e3-4565-ae5b-7324e4c01ef0 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:31.859944 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 08:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:59:31.867119 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868c23f-5d93-4838-b3b1-72a14565feb8 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:31.868225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868c23f-5d93-4838-b3b1-72a14565feb8 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:31.868364 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868c23f-5d93-4838-b3b1-72a14565feb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:31.962635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-4868c23f-5d93-4838-b3b1-72a14565feb8 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:31.964769 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4868c23f-5d93-4838-b3b1-72a14565feb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:32.083772 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 08:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:59:32.251713 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852bd936-08e3-4565-ae5b-7324e4c01ef0 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:32.254035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-852bd936-08e3-4565-ae5b-7324e4c01ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:32.309934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:32.429558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852bd936-08e3-4565-ae5b-7324e4c01ef0 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:32.583810 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010db39-564a-4e1b-83b4-ce63b39de1ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:32.584344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010db39-564a-4e1b-83b4-ce63b39de1ea service nova] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:32.584691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010db39-564a-4e1b-83b4-ce63b39de1ea service nova] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:32.585088 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010db39-564a-4e1b-83b4-ce63b39de1ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:32.587339 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-d010db39-564a-4e1b-83b4-ce63b39de1ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:32.595137 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:32.689476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010db39-564a-4e1b-83b4-ce63b39de1ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:32.691122 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010db39-564a-4e1b-83b4-ce63b39de1ea service nova] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:32.715253 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 711/1431] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:59:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:59:33.646319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e601e-6120-4276-a123-14d018a8ab9f None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:33.649732 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e601e-6120-4276-a123-14d018a8ab9f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:33.650170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e601e-6120-4276-a123-14d018a8ab9f None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:33.651747 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 08:59:33 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:59:33.658151 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829275e-d8d4-405b-94b9-5942188e316c None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:33.658803 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829275e-d8d4-405b-94b9-5942188e316c None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:33.658803 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c829275e-d8d4-405b-94b9-5942188e316c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:33.748882 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-c829275e-d8d4-405b-94b9-5942188e316c None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:33.750151 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-c829275e-d8d4-405b-94b9-5942188e316c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:33.850119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:33.888158 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 08:59:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:59:34.055410 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e601e-6120-4276-a123-14d018a8ab9f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:34.056982 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f97e601e-6120-4276-a123-14d018a8ab9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:34.077956 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:34.254618 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e601e-6120-4276-a123-14d018a8ab9f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:34.357547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:34.386175 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea510b5c-df6e-406d-92a3-82e812d1d610 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:34.386858 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea510b5c-df6e-406d-92a3-82e812d1d610 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:34.387184 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea510b5c-df6e-406d-92a3-82e812d1d610 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:34.387525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea510b5c-df6e-406d-92a3-82e812d1d610 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:34.389388 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ea510b5c-df6e-406d-92a3-82e812d1d610 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:34.491511 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea510b5c-df6e-406d-92a3-82e812d1d610 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:34.493688 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea510b5c-df6e-406d-92a3-82e812d1d610 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:34.514623 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 722/1434] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:59:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:59:35.413674 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba89cf-46ca-4d8d-91ff-f62773a5708b None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:35.414312 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba89cf-46ca-4d8d-91ff-f62773a5708b None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:35.414312 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba89cf-46ca-4d8d-91ff-f62773a5708b None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:35.415642 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 08:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:59:35.420936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1c1cc-0e1b-4a44-a865-bc41c56f3a7f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:35.421581 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1c1cc-0e1b-4a44-a865-bc41c56f3a7f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:35.421581 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f1c1cc-0e1b-4a44-a865-bc41c56f3a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:35.504362 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-12f1c1cc-0e1b-4a44-a865-bc41c56f3a7f None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:35.505600 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-12f1c1cc-0e1b-4a44-a865-bc41c56f3a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:35.551739 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:35.614208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 08:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 08:59:35.769971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba89cf-46ca-4d8d-91ff-f62773a5708b None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:35.771407 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-1eba89cf-46ca-4d8d-91ff-f62773a5708b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:35.965577 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba89cf-46ca-4d8d-91ff-f62773a5708b None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:36.022082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:36.094669 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d471508-a8ef-4103-8c79-0459fa48ec75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:36.095637 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d471508-a8ef-4103-8c79-0459fa48ec75 service nova] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:36.096119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d471508-a8ef-4103-8c79-0459fa48ec75 service nova] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:36.096600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d471508-a8ef-4103-8c79-0459fa48ec75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:36.099613 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-8d471508-a8ef-4103-8c79-0459fa48ec75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:36.229408 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d471508-a8ef-4103-8c79-0459fa48ec75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:36.231455 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d471508-a8ef-4103-8c79-0459fa48ec75 service nova] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:36.252183 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 714/1437] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:59:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:59:37.171103 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677732f-a02b-450b-9da7-24a618924408 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:37.171676 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677732f-a02b-450b-9da7-24a618924408 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:37.172059 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677732f-a02b-450b-9da7-24a618924408 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:37.174165 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 08:59:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:59:37.181119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bce3ca-ee7c-4a75-ba52-e6ff4ce855df None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:37.181672 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bce3ca-ee7c-4a75-ba52-e6ff4ce855df None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:37.181801 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bce3ca-ee7c-4a75-ba52-e6ff4ce855df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:37.268886 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-e6bce3ca-ee7c-4a75-ba52-e6ff4ce855df None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:37.270107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e6bce3ca-ee7c-4a75-ba52-e6ff4ce855df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:37.272330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:37.396751 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 08:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:59:37.559452 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677732f-a02b-450b-9da7-24a618924408 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:37.561717 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9677732f-a02b-450b-9da7-24a618924408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:37.734254 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677732f-a02b-450b-9da7-24a618924408 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:37.771168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:37.871456 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587e253-a50d-48f1-aea4-2b4b9d974cb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:37.872229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587e253-a50d-48f1-aea4-2b4b9d974cb8 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:37.872708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587e253-a50d-48f1-aea4-2b4b9d974cb8 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:37.873196 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587e253-a50d-48f1-aea4-2b4b9d974cb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:37.875975 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8587e253-a50d-48f1-aea4-2b4b9d974cb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:38.024665 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8587e253-a50d-48f1-aea4-2b4b9d974cb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:38.026728 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8587e253-a50d-48f1-aea4-2b4b9d974cb8 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:38.047933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 725/1440] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:59:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 08:59:40.047702 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d67c5-3118-49c8-a9be-260cd64700d4 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:40.048304 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d67c5-3118-49c8-a9be-260cd64700d4 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:40.048304 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707d67c5-3118-49c8-a9be-260cd64700d4 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:40.049666 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 08:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:59:40.055374 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86d2af-d624-442a-9ee5-11a9260c083b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:40.055926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86d2af-d624-442a-9ee5-11a9260c083b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:40.056068 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86d2af-d624-442a-9ee5-11a9260c083b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:40.112074 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:40.151291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9f86d2af-d624-442a-9ee5-11a9260c083b None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:40.152645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9f86d2af-d624-442a-9ee5-11a9260c083b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:40.283855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 08:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:59:40.326043 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d67c5-3118-49c8-a9be-260cd64700d4 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:40.327920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-707d67c5-3118-49c8-a9be-260cd64700d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:40.447155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707d67c5-3118-49c8-a9be-260cd64700d4 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:40.503820 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:40.557324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98e7a9-3f53-4c50-ba70-82b274febf99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:40.557857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98e7a9-3f53-4c50-ba70-82b274febf99 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:40.558303 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98e7a9-3f53-4c50-ba70-82b274febf99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:40.558582 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98e7a9-3f53-4c50-ba70-82b274febf99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:40.560226 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-3f98e7a9-3f53-4c50-ba70-82b274febf99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:40.696802 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98e7a9-3f53-4c50-ba70-82b274febf99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:40.698319 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98e7a9-3f53-4c50-ba70-82b274febf99 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:40.722647 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 717/1443] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:59:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:59:43.161388 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99eb0c-63c1-4d79-a97b-b3373bb7cefc None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:43.162034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99eb0c-63c1-4d79-a97b-b3373bb7cefc None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:43.162034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f99eb0c-63c1-4d79-a97b-b3373bb7cefc None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:43.163358 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 08:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:59:43.172579 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a0574-fda9-43b2-a8ab-b02125b854bc None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:43.173084 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a0574-fda9-43b2-a8ab-b02125b854bc None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:43.173287 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699a0574-fda9-43b2-a8ab-b02125b854bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:43.245788 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-699a0574-fda9-43b2-a8ab-b02125b854bc None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:43.247235 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-699a0574-fda9-43b2-a8ab-b02125b854bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:43.262723 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:43.352379 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 08:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 08:59:43.391329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99eb0c-63c1-4d79-a97b-b3373bb7cefc None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:43.392891 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0f99eb0c-63c1-4d79-a97b-b3373bb7cefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:43.508678 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f99eb0c-63c1-4d79-a97b-b3373bb7cefc None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:43.522171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:43.592999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e215a99-3a1d-403b-a5e2-bbdaa5e1ee4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:43.593615 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e215a99-3a1d-403b-a5e2-bbdaa5e1ee4c service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:43.593961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e215a99-3a1d-403b-a5e2-bbdaa5e1ee4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:43.594199 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e215a99-3a1d-403b-a5e2-bbdaa5e1ee4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:43.595834 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7e215a99-3a1d-403b-a5e2-bbdaa5e1ee4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:43.750655 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e215a99-3a1d-403b-a5e2-bbdaa5e1ee4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:43.752285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e215a99-3a1d-403b-a5e2-bbdaa5e1ee4c service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:43.777608 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 728/1446] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:59:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:59:46.286914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a907f-a106-4aab-8ba5-38cbae1111ea None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:46.287476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a907f-a106-4aab-8ba5-38cbae1111ea None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:46.287476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a907f-a106-4aab-8ba5-38cbae1111ea None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:46.288799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 08:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:59:46.295745 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805e235-8d09-4217-8e7c-4aa8643020fc None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:46.296587 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805e235-8d09-4217-8e7c-4aa8643020fc None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:46.296587 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805e235-8d09-4217-8e7c-4aa8643020fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:46.367948 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:46.390762 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-7805e235-8d09-4217-8e7c-4aa8643020fc None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:46.391988 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7805e235-8d09-4217-8e7c-4aa8643020fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:46.527321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 08:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:59:46.573301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a907f-a106-4aab-8ba5-38cbae1111ea None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:46.575133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-408a907f-a106-4aab-8ba5-38cbae1111ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:46.700522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a907f-a106-4aab-8ba5-38cbae1111ea None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:46.746228 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:46.794985 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99966d55-9f54-47ad-ad73-da82fd1b1bba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:46.795535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99966d55-9f54-47ad-ad73-da82fd1b1bba service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:46.796020 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99966d55-9f54-47ad-ad73-da82fd1b1bba service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:46.796247 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99966d55-9f54-47ad-ad73-da82fd1b1bba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:46.797959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-99966d55-9f54-47ad-ad73-da82fd1b1bba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:46.949639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99966d55-9f54-47ad-ad73-da82fd1b1bba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:46.952749 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99966d55-9f54-47ad-ad73-da82fd1b1bba service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:46.990984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 720/1449] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:59:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:59:49.267354 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88e20f-abb2-4954-8855-3cf407caf0c2 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:49.267905 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88e20f-abb2-4954-8855-3cf407caf0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:49.267905 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88e20f-abb2-4954-8855-3cf407caf0c2 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:49.269244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 08:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:59:49.276036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cdc89a-437d-4aaa-87eb-f754d1557e9a None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:49.276572 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cdc89a-437d-4aaa-87eb-f754d1557e9a None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:49.276858 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cdc89a-437d-4aaa-87eb-f754d1557e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:49.356368 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-a5cdc89a-437d-4aaa-87eb-f754d1557e9a None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:49.357594 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-a5cdc89a-437d-4aaa-87eb-f754d1557e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:49.381046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:49.478180 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 08:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 08:59:49.519034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88e20f-abb2-4954-8855-3cf407caf0c2 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:49.520512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ce88e20f-abb2-4954-8855-3cf407caf0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:49.639095 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88e20f-abb2-4954-8855-3cf407caf0c2 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:49.668236 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:49.728677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509fe04-c42c-4c55-9eca-25350f35ab60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:49.729302 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509fe04-c42c-4c55-9eca-25350f35ab60 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:49.729580 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509fe04-c42c-4c55-9eca-25350f35ab60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:49.729854 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509fe04-c42c-4c55-9eca-25350f35ab60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:49.731520 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1509fe04-c42c-4c55-9eca-25350f35ab60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:49.865568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509fe04-c42c-4c55-9eca-25350f35ab60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:49.866977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1509fe04-c42c-4c55-9eca-25350f35ab60 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:49.887624 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 731/1452] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:59:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:59:52.122734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38000c6b-28c9-445e-96a6-01d026a29591 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:52.123337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38000c6b-28c9-445e-96a6-01d026a29591 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:52.123337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38000c6b-28c9-445e-96a6-01d026a29591 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:52.124831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 08:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:59:52.133327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbc48b-0d41-4f35-8d43-46b192835cbd None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:52.133972 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbc48b-0d41-4f35-8d43-46b192835cbd None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:52.134317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bbc48b-0d41-4f35-8d43-46b192835cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:52.211577 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:52.214456 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-11bbc48b-0d41-4f35-8d43-46b192835cbd None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:52.215685 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-11bbc48b-0d41-4f35-8d43-46b192835cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:52.330489 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 08:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:59:52.370512 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38000c6b-28c9-445e-96a6-01d026a29591 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:52.372360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-38000c6b-28c9-445e-96a6-01d026a29591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:52.496260 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38000c6b-28c9-445e-96a6-01d026a29591 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:52.554614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:52.583961 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf24fb7-63fe-4730-af05-960adda0cfa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:52.584601 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf24fb7-63fe-4730-af05-960adda0cfa9 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:52.584951 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf24fb7-63fe-4730-af05-960adda0cfa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:52.585279 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf24fb7-63fe-4730-af05-960adda0cfa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:52.587722 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-9cf24fb7-63fe-4730-af05-960adda0cfa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:52.717959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf24fb7-63fe-4730-af05-960adda0cfa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:52.719463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf24fb7-63fe-4730-af05-960adda0cfa9 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:52.745605 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 723/1455] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:59:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 08:59:54.880640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e6f56-f3d4-4228-a518-f1fa32dba9f0 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:54.881218 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e6f56-f3d4-4228-a518-f1fa32dba9f0 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:54.881218 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e6f56-f3d4-4228-a518-f1fa32dba9f0 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:54.882612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 08:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 08:59:54.888139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000b393-d6be-4a1d-99d9-f24e5ee750a1 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:54.888610 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000b393-d6be-4a1d-99d9-f24e5ee750a1 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:54.888717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000b393-d6be-4a1d-99d9-f24e5ee750a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:54.965191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-6000b393-d6be-4a1d-99d9-f24e5ee750a1 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:54.966446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-6000b393-d6be-4a1d-99d9-f24e5ee750a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:54.998840 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:55.075002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 08:59:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:59:55.114502 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e6f56-f3d4-4228-a518-f1fa32dba9f0 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:55.116054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-048e6f56-f3d4-4228-a518-f1fa32dba9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:55.226614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e6f56-f3d4-4228-a518-f1fa32dba9f0 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:55.255908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:55.305564 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fb62b-2d51-42cc-81eb-c763dd54ff8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:55.306220 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fb62b-2d51-42cc-81eb-c763dd54ff8a service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:55.306499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fb62b-2d51-42cc-81eb-c763dd54ff8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:55.306771 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fb62b-2d51-42cc-81eb-c763dd54ff8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:55.308413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-299fb62b-2d51-42cc-81eb-c763dd54ff8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:55.425864 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299fb62b-2d51-42cc-81eb-c763dd54ff8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:55.427390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299fb62b-2d51-42cc-81eb-c763dd54ff8a service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:55.447173 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 734/1458] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:59:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 08:59:57.592213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a3775-2c66-418a-90ef-98c392cfd55c None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:57.592908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a3775-2c66-418a-90ef-98c392cfd55c None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:57.593269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a3775-2c66-418a-90ef-98c392cfd55c None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:57.595357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 08:59:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 08:59:57.601521 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c22610-de05-4940-9864-9d433f096e7f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:57.602057 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c22610-de05-4940-9864-9d433f096e7f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:57.602164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c22610-de05-4940-9864-9d433f096e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:57.696170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-e1c22610-de05-4940-9864-9d433f096e7f None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 08:59:57.697408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e1c22610-de05-4940-9864-9d433f096e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:57.707299 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:57.813193 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 08:59:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 08:59:57.852952 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a3775-2c66-418a-90ef-98c392cfd55c None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 08:59:57.854501 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-0f5a3775-2c66-418a-90ef-98c392cfd55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:57.988467 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a3775-2c66-418a-90ef-98c392cfd55c None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 08:59:58.049507 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 08:59:58.069225 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db43535d-84bc-4064-8940-793a3c686c30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 08:59:58.070409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43535d-84bc-4064-8940-793a3c686c30 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 08:59:58.070830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43535d-84bc-4064-8940-793a3c686c30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 08:59:58.071108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43535d-84bc-4064-8940-793a3c686c30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 08:59:58.072755 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-db43535d-84bc-4064-8940-793a3c686c30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 08:59:58.200696 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db43535d-84bc-4064-8940-793a3c686c30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 08:59:58.202488 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db43535d-84bc-4064-8940-793a3c686c30 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 08:59:58.223266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 726/1461] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 08:59:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:00:00.375416 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c03502-cf49-4d27-873a-f67da05bdf5f None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:00.376144 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c03502-cf49-4d27-873a-f67da05bdf5f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:00.376144 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c03502-cf49-4d27-873a-f67da05bdf5f None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:00.377342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:00:00.383907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818abdbc-7636-496a-8b0a-e97e2cdd7242 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:00.384421 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818abdbc-7636-496a-8b0a-e97e2cdd7242 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:00.384631 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818abdbc-7636-496a-8b0a-e97e2cdd7242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:00.460222 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-818abdbc-7636-496a-8b0a-e97e2cdd7242 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:00:00.462148 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-818abdbc-7636-496a-8b0a-e97e2cdd7242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:00.511464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:00.578658 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:00:00.619923 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c03502-cf49-4d27-873a-f67da05bdf5f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:00.621970 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-15c03502-cf49-4d27-873a-f67da05bdf5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:00.754510 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c03502-cf49-4d27-873a-f67da05bdf5f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:00.803206 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:00.850078 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e10a8-4943-4c81-b55e-4ffa501d6b04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:00.851777 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e10a8-4943-4c81-b55e-4ffa501d6b04 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:00.852294 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e10a8-4943-4c81-b55e-4ffa501d6b04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:00.852807 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e10a8-4943-4c81-b55e-4ffa501d6b04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:00.854707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f47e10a8-4943-4c81-b55e-4ffa501d6b04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:01.009204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e10a8-4943-4c81-b55e-4ffa501d6b04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:01.010612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e10a8-4943-4c81-b55e-4ffa501d6b04 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:01.034204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 737/1464] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:00:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:00:03.167697 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c90406-2642-44eb-838b-d098d0eb0020 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:03.168280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c90406-2642-44eb-838b-d098d0eb0020 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:03.168377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c90406-2642-44eb-838b-d098d0eb0020 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:03.170595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:00:03.175368 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758095e-85fa-4820-a141-da4d249c524d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:03.176645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758095e-85fa-4820-a141-da4d249c524d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:03.176645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0758095e-85fa-4820-a141-da4d249c524d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:03.254115 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-0758095e-85fa-4820-a141-da4d249c524d None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:00:03.255357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0758095e-85fa-4820-a141-da4d249c524d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:03.276912 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:03.365835 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:00:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:00:03.404757 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c90406-2642-44eb-838b-d098d0eb0020 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:03.406411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-b2c90406-2642-44eb-838b-d098d0eb0020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:03.526306 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c90406-2642-44eb-838b-d098d0eb0020 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:03.601663 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:03.608751 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30556912-8d94-4a83-aa2d-008006a78124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:03.609376 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30556912-8d94-4a83-aa2d-008006a78124 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:03.609663 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30556912-8d94-4a83-aa2d-008006a78124 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:03.609967 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30556912-8d94-4a83-aa2d-008006a78124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:03.611604 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-30556912-8d94-4a83-aa2d-008006a78124 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:03.746391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30556912-8d94-4a83-aa2d-008006a78124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:03.747919 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30556912-8d94-4a83-aa2d-008006a78124 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:03.768656 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 729/1467] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:00:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:00:05.892740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86c6aa-8aa4-4bab-837b-bed28b8adb01 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:05.893372 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86c6aa-8aa4-4bab-837b-bed28b8adb01 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:05.893372 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86c6aa-8aa4-4bab-837b-bed28b8adb01 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:05.894766 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:00:05.901110 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17301ddc-eb22-4217-9d54-8ebc7acd9f34 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:05.906771 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17301ddc-eb22-4217-9d54-8ebc7acd9f34 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:05.906771 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17301ddc-eb22-4217-9d54-8ebc7acd9f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:05.980559 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-17301ddc-eb22-4217-9d54-8ebc7acd9f34 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:00:05.981829 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-17301ddc-eb22-4217-9d54-8ebc7acd9f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:06.039478 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:06.095753 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:00:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:00:06.135723 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86c6aa-8aa4-4bab-837b-bed28b8adb01 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:06.137555 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4c86c6aa-8aa4-4bab-837b-bed28b8adb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:06.296813 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86c6aa-8aa4-4bab-837b-bed28b8adb01 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:06.351142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:06.379606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b646d-2ae0-4d29-af4c-454bb2422970 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:06.380237 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b646d-2ae0-4d29-af4c-454bb2422970 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:06.380521 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b646d-2ae0-4d29-af4c-454bb2422970 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:06.380800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b646d-2ae0-4d29-af4c-454bb2422970 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:06.382518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1c5b646d-2ae0-4d29-af4c-454bb2422970 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:06.527208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b646d-2ae0-4d29-af4c-454bb2422970 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:06.528664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b646d-2ae0-4d29-af4c-454bb2422970 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:06.549407 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 740/1470] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:00:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:00:16.093631 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=22 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:16.546748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=66 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:26.783519 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0cadce-97cd-49de-8718-a30d58e9092a None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:26.784271 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0cadce-97cd-49de-8718-a30d58e9092a None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:26.784271 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0cadce-97cd-49de-8718-a30d58e9092a None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:26.785740 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 731/1471] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 09:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:00:26.791640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc2291-3df9-4884-b713-16303f53a040 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:26.792167 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc2291-3df9-4884-b713-16303f53a040 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:26.792277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc2291-3df9-4884-b713-16303f53a040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:26.864769 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-14bc2291-3df9-4884-b713-16303f53a040 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:00:26.865954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-14bc2291-3df9-4884-b713-16303f53a040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:26.977094 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 741/1472] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 09:00:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:00:27.075241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a18e67-42c8-4717-a239-7dc5687f1d58 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:27.075522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a18e67-42c8-4717-a239-7dc5687f1d58 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:27.075839 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a18e67-42c8-4717-a239-7dc5687f1d58 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:27.077444 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 732/1473] 10.222.0.46 () {54 vars in 966 bytes} [Tue Aug 2 09:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:00:27.082893 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ba8f5-0e40-4eb1-8283-bc0d185ccf57 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:27.083329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ba8f5-0e40-4eb1-8283-bc0d185ccf57 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:27.083616 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ba8f5-0e40-4eb1-8283-bc0d185ccf57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:27.284777 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-358ba8f5-0e40-4eb1-8283-bc0d185ccf57 None None] MFA Rules not processed for user `4abd1512ad224da896c6f2257f9602e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:00:27.286044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-358ba8f5-0e40-4eb1-8283-bc0d185ccf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:27.308378 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:27.366491 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 742/1474] 10.222.0.46 () {58 vars in 1063 bytes} [Tue Aug 2 09:00:27 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:00:27.374795 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a18e67-42c8-4717-a239-7dc5687f1d58 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:27.376339 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-c1a18e67-42c8-4717-a239-7dc5687f1d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:27.513210 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a18e67-42c8-4717-a239-7dc5687f1d58 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:27.594131 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e7107-6cab-4c21-a628-8996262f5bf9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4abd1512ad224da896c6f2257f9602e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:27.594647 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e7107-6cab-4c21-a628-8996262f5bf9 service glance] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:27.597754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e7107-6cab-4c21-a628-8996262f5bf9 service glance] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:27.597754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e7107-6cab-4c21-a628-8996262f5bf9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:27.597754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-d93e7107-6cab-4c21-a628-8996262f5bf9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:27.655329 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:27.720894 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e7107-6cab-4c21-a628-8996262f5bf9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:27.722849 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e7107-6cab-4c21-a628-8996262f5bf9 service glance] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:27.743572 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 733/1475] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:00:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:00:29.877696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91310afa-4506-4d43-9884-96f2da14ce15 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:29.878338 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91310afa-4506-4d43-9884-96f2da14ce15 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:29.878338 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91310afa-4506-4d43-9884-96f2da14ce15 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:29.879630 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 743/1476] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 09:00:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:00:29.885291 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ef53c-7462-4574-9258-16fd04ccdf5e None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:29.885738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ef53c-7462-4574-9258-16fd04ccdf5e None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:29.885921 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ef53c-7462-4574-9258-16fd04ccdf5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:29.956630 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-b49ef53c-7462-4574-9258-16fd04ccdf5e None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:00:29.957866 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-b49ef53c-7462-4574-9258-16fd04ccdf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:30.062379 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 734/1477] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 09:00:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:00:30.086765 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91310afa-4506-4d43-9884-96f2da14ce15 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:30.088287 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-91310afa-4506-4d43-9884-96f2da14ce15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:30.218489 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91310afa-4506-4d43-9884-96f2da14ce15 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:30.269514 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:30.316351 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444705c1-eb12-405f-9bd3-6ebce1a2c97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:30.316981 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444705c1-eb12-405f-9bd3-6ebce1a2c97c admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:30.317273 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444705c1-eb12-405f-9bd3-6ebce1a2c97c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:30.317551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444705c1-eb12-405f-9bd3-6ebce1a2c97c admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:30.318285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444705c1-eb12-405f-9bd3-6ebce1a2c97c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:30.320059 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: WARNING py.warnings [None req-444705c1-eb12-405f-9bd3-6ebce1a2c97c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:00:30.320059 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: warnings.warn(msg) Aug 02 09:00:30.320059 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: Aug 02 09:00:30.321234 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444705c1-eb12-405f-9bd3-6ebce1a2c97c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:30.340873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 744/1478] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 2 09:00:30 2022] GET /identity/v3/projects => generated 2553 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:00:32.449420 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad78a84-8261-4e95-a10c-b8a11ea4904d None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:32.450090 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad78a84-8261-4e95-a10c-b8a11ea4904d None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:32.450090 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad78a84-8261-4e95-a10c-b8a11ea4904d None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:32.451662 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 735/1479] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 09:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:00:32.457524 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eee886-25c6-41c5-bd49-d5e96c1d36cd None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:32.458088 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eee886-25c6-41c5-bd49-d5e96c1d36cd None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:32.458088 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eee886-25c6-41c5-bd49-d5e96c1d36cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:32.537820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-b6eee886-25c6-41c5-bd49-d5e96c1d36cd None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:00:32.539126 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b6eee886-25c6-41c5-bd49-d5e96c1d36cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:32.646720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 745/1480] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 09:00:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:00:32.691948 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad78a84-8261-4e95-a10c-b8a11ea4904d None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:32.693664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-1ad78a84-8261-4e95-a10c-b8a11ea4904d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:32.705551 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:32.859393 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad78a84-8261-4e95-a10c-b8a11ea4904d None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:32.951268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:32.985319 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b952b-74fe-4715-b2e8-284b2bc58740 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:32.985858 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b952b-74fe-4715-b2e8-284b2bc58740 service nova] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:32.986172 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b952b-74fe-4715-b2e8-284b2bc58740 service nova] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:32.986449 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b952b-74fe-4715-b2e8-284b2bc58740 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:32.988105 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-672b952b-74fe-4715-b2e8-284b2bc58740 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:33.084458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b952b-74fe-4715-b2e8-284b2bc58740 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:33.086366 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b952b-74fe-4715-b2e8-284b2bc58740 service nova] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:33.106631 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 736/1481] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:00:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:00:35.365786 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3449d-95bf-45a9-a561-f7d96101e483 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:35.366932 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3449d-95bf-45a9-a561-f7d96101e483 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:35.366932 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3449d-95bf-45a9-a561-f7d96101e483 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:35.369041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 746/1482] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 09:00:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:00:35.374803 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236e300-cd68-4541-8302-a139773bbcc6 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:35.375312 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236e300-cd68-4541-8302-a139773bbcc6 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:35.375434 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236e300-cd68-4541-8302-a139773bbcc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:35.450610 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-0236e300-cd68-4541-8302-a139773bbcc6 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:00:35.452136 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-0236e300-cd68-4541-8302-a139773bbcc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:35.508384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:35.568484 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 737/1483] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 09:00:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:00:35.614897 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3449d-95bf-45a9-a561-f7d96101e483 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:35.617107 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-86c3449d-95bf-45a9-a561-f7d96101e483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:35.642055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:35.736565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3449d-95bf-45a9-a561-f7d96101e483 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:35.826268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee20a85-9f25-4b15-897c-110a445d2abd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:35.827147 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee20a85-9f25-4b15-897c-110a445d2abd service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:35.827430 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee20a85-9f25-4b15-897c-110a445d2abd service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:35.827704 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee20a85-9f25-4b15-897c-110a445d2abd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:35.829366 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2ee20a85-9f25-4b15-897c-110a445d2abd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:35.908045 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:35.945664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee20a85-9f25-4b15-897c-110a445d2abd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:35.947212 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee20a85-9f25-4b15-897c-110a445d2abd service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:35.967063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 747/1484] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:00:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:00:38.064526 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedb176-0aaf-4cc3-a28b-cef0b4472df3 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:38.065066 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedb176-0aaf-4cc3-a28b-cef0b4472df3 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:38.065066 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dedb176-0aaf-4cc3-a28b-cef0b4472df3 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:38.066436 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 738/1485] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 09:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:00:38.072404 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c4bb0-1a2c-4bbb-acb8-f844ff1d4a6a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:38.072835 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c4bb0-1a2c-4bbb-acb8-f844ff1d4a6a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:38.072966 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c4bb0-1a2c-4bbb-acb8-f844ff1d4a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:38.145569 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-688c4bb0-1a2c-4bbb-acb8-f844ff1d4a6a None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:00:38.146871 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-688c4bb0-1a2c-4bbb-acb8-f844ff1d4a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:38.259458 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 748/1486] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 09:00:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:00:38.297663 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dedb176-0aaf-4cc3-a28b-cef0b4472df3 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:38.299349 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-5dedb176-0aaf-4cc3-a28b-cef0b4472df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:38.415640 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dedb176-0aaf-4cc3-a28b-cef0b4472df3 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:38.492509 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:38.494719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d0347-d7db-4586-b25b-cdfa810e16ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:38.495335 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d0347-d7db-4586-b25b-cdfa810e16ea service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:38.495612 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d0347-d7db-4586-b25b-cdfa810e16ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:38.495913 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d0347-d7db-4586-b25b-cdfa810e16ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:38.497570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-be7d0347-d7db-4586-b25b-cdfa810e16ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:38.623260 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d0347-d7db-4586-b25b-cdfa810e16ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:38.624699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d0347-d7db-4586-b25b-cdfa810e16ea service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:38.643540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 739/1487] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:00:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:00:40.805868 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45838e91-b166-412d-a31a-4161727c96b9 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:40.806503 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45838e91-b166-412d-a31a-4161727c96b9 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:40.806503 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45838e91-b166-412d-a31a-4161727c96b9 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:40.807877 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 749/1488] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 09:00:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:00:40.814247 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56720cda-6bad-406a-96af-2872d90333f4 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:40.814714 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56720cda-6bad-406a-96af-2872d90333f4 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:40.815196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56720cda-6bad-406a-96af-2872d90333f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:40.894727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-56720cda-6bad-406a-96af-2872d90333f4 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:00:40.896003 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-56720cda-6bad-406a-96af-2872d90333f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:40.964885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:41.028326 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 740/1489] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 09:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:00:41.071166 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45838e91-b166-412d-a31a-4161727c96b9 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:41.072716 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-45838e91-b166-412d-a31a-4161727c96b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:41.115544 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:41.189872 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45838e91-b166-412d-a31a-4161727c96b9 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:41.268424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a699c23-f6b1-4b5d-8ba5-b99e8d85031e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:41.269053 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a699c23-f6b1-4b5d-8ba5-b99e8d85031e service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:41.269336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a699c23-f6b1-4b5d-8ba5-b99e8d85031e service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:41.269612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a699c23-f6b1-4b5d-8ba5-b99e8d85031e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:41.271405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6a699c23-f6b1-4b5d-8ba5-b99e8d85031e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:41.360137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:41.389519 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a699c23-f6b1-4b5d-8ba5-b99e8d85031e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:41.391019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a699c23-f6b1-4b5d-8ba5-b99e8d85031e service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:41.410785 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 750/1490] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:00:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:00:44.075667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ee4ec-6d18-4b21-8492-b9a65a650864 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:44.076337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ee4ec-6d18-4b21-8492-b9a65a650864 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:44.076337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ee4ec-6d18-4b21-8492-b9a65a650864 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:44.077953 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 741/1491] 10.222.0.46 () {54 vars in 926 bytes} [Tue Aug 2 09:00:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:00:44.083467 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d6660-86be-42cb-aba7-0624a2e74887 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:44.083900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d6660-86be-42cb-aba7-0624a2e74887 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:44.084020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d6660-86be-42cb-aba7-0624a2e74887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:44.159167 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-be2d6660-86be-42cb-aba7-0624a2e74887 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:00:44.160379 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-be2d6660-86be-42cb-aba7-0624a2e74887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:44.259603 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 751/1492] 10.222.0.46 () {58 vars in 1023 bytes} [Tue Aug 2 09:00:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:00:44.299540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ee4ec-6d18-4b21-8492-b9a65a650864 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:00:44.301141 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f65ee4ec-6d18-4b21-8492-b9a65a650864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:44.425090 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ee4ec-6d18-4b21-8492-b9a65a650864 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:00:44.513763 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910b76a-c4c1-4363-8233-589448b9124f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:00:44.514322 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910b76a-c4c1-4363-8233-589448b9124f service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:00:44.514605 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910b76a-c4c1-4363-8233-589448b9124f service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:00:44.514877 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910b76a-c4c1-4363-8233-589448b9124f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:00:44.516498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-c910b76a-c4c1-4363-8233-589448b9124f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:00:44.525928 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:44.629241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910b76a-c4c1-4363-8233-589448b9124f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:00:44.630720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910b76a-c4c1-4363-8233-589448b9124f service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:00:44.651472 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 742/1493] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:00:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:00:54.257789 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=82 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:00:54.648974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=50 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:25.011386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edaf8dd-95fd-4b16-9633-a368110dcfa5 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:25.011975 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edaf8dd-95fd-4b16-9633-a368110dcfa5 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:25.011975 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edaf8dd-95fd-4b16-9633-a368110dcfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:25.095147 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-6edaf8dd-95fd-4b16-9633-a368110dcfa5 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:02:25.096301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6edaf8dd-95fd-4b16-9633-a368110dcfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:25.204854 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 752/1494] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:02:25.212998 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afe50f-854d-45f6-8ab8-6a14695c2476 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:25.213589 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afe50f-854d-45f6-8ab8-6a14695c2476 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:25.213589 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afe50f-854d-45f6-8ab8-6a14695c2476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:25.234262 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-41afe50f-854d-45f6-8ab8-6a14695c2476 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:02:25.235630 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-41afe50f-854d-45f6-8ab8-6a14695c2476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:25.245204 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 743/1495] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:02:25.250001 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edaf8dd-95fd-4b16-9633-a368110dcfa5 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:02:25.260131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edaf8dd-95fd-4b16-9633-a368110dcfa5 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:02:25.264910 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134de51-fa58-4011-a106-ed4409d7bac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:02:25.265596 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134de51-fa58-4011-a106-ed4409d7bac1 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:25.265878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134de51-fa58-4011-a106-ed4409d7bac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:25.266182 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c134de51-fa58-4011-a106-ed4409d7bac1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:25.266912 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134de51-fa58-4011-a106-ed4409d7bac1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:02:25.267217 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134de51-fa58-4011-a106-ed4409d7bac1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:02:25.269823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: WARNING py.warnings [None req-c134de51-fa58-4011-a106-ed4409d7bac1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:02:25.269823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: warnings.warn(msg) Aug 02 09:02:25.269823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: Aug 02 09:02:25.269823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c134de51-fa58-4011-a106-ed4409d7bac1 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:02:25.280755 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 753/1496] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:02:25.289611 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41afe50f-854d-45f6-8ab8-6a14695c2476 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:02:25.299349 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41afe50f-854d-45f6-8ab8-6a14695c2476 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:02:25.304834 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f12e8-a904-4e22-8481-d1bf2fb9267c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:02:25.305547 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f12e8-a904-4e22-8481-d1bf2fb9267c admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:25.305815 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f12e8-a904-4e22-8481-d1bf2fb9267c admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:25.306124 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f12e8-a904-4e22-8481-d1bf2fb9267c admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:25.306802 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f12e8-a904-4e22-8481-d1bf2fb9267c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:02:25.308603 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: WARNING py.warnings [None req-095f12e8-a904-4e22-8481-d1bf2fb9267c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:02:25.308603 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: warnings.warn(msg) Aug 02 09:02:25.308603 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: Aug 02 09:02:25.309803 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f12e8-a904-4e22-8481-d1bf2fb9267c admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:02:25.437494 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 744/1497] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:02:25 2022] POST /identity/v3/projects => generated 375 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:02:25.442283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134de51-fa58-4011-a106-ed4409d7bac1 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:02:25.453169 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c134de51-fa58-4011-a106-ed4409d7bac1 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:02:25.510301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:25.517428 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129660d5-a3ae-4379-9429-8dbbdeb21359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:02:25.518084 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129660d5-a3ae-4379-9429-8dbbdeb21359 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:25.518371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129660d5-a3ae-4379-9429-8dbbdeb21359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:25.518701 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129660d5-a3ae-4379-9429-8dbbdeb21359 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:25.519891 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129660d5-a3ae-4379-9429-8dbbdeb21359 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:02:25.520908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: WARNING py.warnings [None req-129660d5-a3ae-4379-9429-8dbbdeb21359 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:02:25.520908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: warnings.warn(msg) Aug 02 09:02:25.520908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: Aug 02 09:02:25.522146 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129660d5-a3ae-4379-9429-8dbbdeb21359 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:02:25.624978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 754/1498] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:02:25 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:02:25.628907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f12e8-a904-4e22-8481-d1bf2fb9267c admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:02:25.641582 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f12e8-a904-4e22-8481-d1bf2fb9267c admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:02:25.649649 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1be096-596b-43b1-90e8-022524cf53f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:02:25.650335 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1be096-596b-43b1-90e8-022524cf53f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:25.650600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1be096-596b-43b1-90e8-022524cf53f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:25.650876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1be096-596b-43b1-90e8-022524cf53f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:25.651581 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1be096-596b-43b1-90e8-022524cf53f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:02:25.652587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: WARNING py.warnings [None req-dd1be096-596b-43b1-90e8-022524cf53f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:02:25.652587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: warnings.warn(msg) Aug 02 09:02:25.652587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: Aug 02 09:02:25.653564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1be096-596b-43b1-90e8-022524cf53f2 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:02:25.668052 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 745/1499] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:02:25.672912 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129660d5-a3ae-4379-9429-8dbbdeb21359 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:02:25.682548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129660d5-a3ae-4379-9429-8dbbdeb21359 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:02:25.687337 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf210fe2-0871-4846-b5cc-450001c74269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:02:25.688050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf210fe2-0871-4846-b5cc-450001c74269 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:25.688239 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf210fe2-0871-4846-b5cc-450001c74269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:25.688518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf210fe2-0871-4846-b5cc-450001c74269 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:25.689210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf210fe2-0871-4846-b5cc-450001c74269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:02:25.690273 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: WARNING py.warnings [None req-cf210fe2-0871-4846-b5cc-450001c74269 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:02:25.690273 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: warnings.warn(msg) Aug 02 09:02:25.690273 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: Aug 02 09:02:25.691157 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf210fe2-0871-4846-b5cc-450001c74269 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:02:25.705591 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 755/1500] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:02:25.712549 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1be096-596b-43b1-90e8-022524cf53f2 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:02:25.724919 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1be096-596b-43b1-90e8-022524cf53f2 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:02:25.729594 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1a1dc-e1b4-495e-bdc3-a803b9ea3971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:02:25.730296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1a1dc-e1b4-495e-bdc3-a803b9ea3971 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:25.730581 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1a1dc-e1b4-495e-bdc3-a803b9ea3971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:25.730860 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1a1dc-e1b4-495e-bdc3-a803b9ea3971 admin admin] PATH_INFO: `/v3/projects/3d800782b5cd44c2819368dd44fe6cf3/users/046dcf4877a745e990cac3cb2c4daaaa/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:25.746544 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc1a1dc-e1b4-495e-bdc3-a803b9ea3971 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d800782b5cd44c2819368dd44fe6cf3, user_id=046dcf4877a745e990cac3cb2c4daaaa, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:02:25.749245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: WARNING py.warnings [None req-cdc1a1dc-e1b4-495e-bdc3-a803b9ea3971 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:02:25.749245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: warnings.warn(msg) Aug 02 09:02:25.749245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: Aug 02 09:02:25.749647 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc1a1dc-e1b4-495e-bdc3-a803b9ea3971 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:02:25.853973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 746/1501] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:02:25 2022] PUT /identity/v3/projects/3d800782b5cd44c2819368dd44fe6cf3/users/046dcf4877a745e990cac3cb2c4daaaa/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:02:25.866498 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d46185-db13-4a4c-889d-971bbd7df344 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:25.866787 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d46185-db13-4a4c-889d-971bbd7df344 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:25.867065 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d46185-db13-4a4c-889d-971bbd7df344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:25.898418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-07d46185-db13-4a4c-889d-971bbd7df344 None None] MFA Rules not processed for user `046dcf4877a745e990cac3cb2c4daaaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:02:25.899690 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-07d46185-db13-4a4c-889d-971bbd7df344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:25.971987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 756/1502] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:02:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:02:25.985683 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6f5f6-0858-4f45-b3e8-ac9bc8544b78 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:25.986137 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6f5f6-0858-4f45-b3e8-ac9bc8544b78 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:25.986561 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6f5f6-0858-4f45-b3e8-ac9bc8544b78 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:25.988573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 747/1503] 10.222.0.46 () {54 vars in 964 bytes} [Tue Aug 2 09:02:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:02:26.001699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad474d14-2af9-4b28-8bab-e0f2ff739b7f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:26.002027 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad474d14-2af9-4b28-8bab-e0f2ff739b7f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:26.002340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad474d14-2af9-4b28-8bab-e0f2ff739b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:26.073259 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-ad474d14-2af9-4b28-8bab-e0f2ff739b7f None None] MFA Rules not processed for user `6e39392b79bc439780415db7c8206098`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:02:26.074497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ad474d14-2af9-4b28-8bab-e0f2ff739b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:26.160937 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 757/1504] 10.222.0.46 () {58 vars in 1061 bytes} [Tue Aug 2 09:02:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:02:26.169346 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6f5f6-0858-4f45-b3e8-ac9bc8544b78 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:02:26.181073 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b6f5f6-0858-4f45-b3e8-ac9bc8544b78 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:02:26.184779 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793287fc-c02f-4578-b054-7e6a30a45c7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:02:26.185394 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793287fc-c02f-4578-b054-7e6a30a45c7e service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:26.185675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793287fc-c02f-4578-b054-7e6a30a45c7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:26.185985 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793287fc-c02f-4578-b054-7e6a30a45c7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:26.195982 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793287fc-c02f-4578-b054-7e6a30a45c7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:02:26.197649 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793287fc-c02f-4578-b054-7e6a30a45c7e service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:02:26.207493 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 748/1505] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:02:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:02:26.266095 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2befed-8fe1-444a-bd5a-0d9d3a5fa514 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:26.266391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2befed-8fe1-444a-bd5a-0d9d3a5fa514 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:26.266672 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2befed-8fe1-444a-bd5a-0d9d3a5fa514 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:26.267950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 758/1506] 10.222.0.46 () {52 vars in 830 bytes} [Tue Aug 2 09:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:02:26.271897 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793287fc-c02f-4578-b054-7e6a30a45c7e service cinder] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:02:26.341650 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793287fc-c02f-4578-b054-7e6a30a45c7e service cinder] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:02:26.347635 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f96473-b629-41d3-b30c-f57650d2bdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:02:26.348355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f96473-b629-41d3-b30c-f57650d2bdc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:26.348640 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f96473-b629-41d3-b30c-f57650d2bdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:26.348921 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f96473-b629-41d3-b30c-f57650d2bdc8 admin admin] PATH_INFO: `/v3/users/046dcf4877a745e990cac3cb2c4daaaa` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:26.350332 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f96473-b629-41d3-b30c-f57650d2bdc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=046dcf4877a745e990cac3cb2c4daaaa)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:02:26.351362 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: WARNING py.warnings [None req-58f96473-b629-41d3-b30c-f57650d2bdc8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:02:26.351362 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: warnings.warn(msg) Aug 02 09:02:26.351362 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: Aug 02 09:02:26.352617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f96473-b629-41d3-b30c-f57650d2bdc8 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:02:26.621916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-58f96473-b629-41d3-b30c-f57650d2bdc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '046dcf4877a745e990cac3cb2c4daaaa'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:02:26.638828 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-58f96473-b629-41d3-b30c-f57650d2bdc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '046dcf4877a745e990cac3cb2c4daaaa'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:02:26.639692 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-58f96473-b629-41d3-b30c-f57650d2bdc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '046dcf4877a745e990cac3cb2c4daaaa'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:02:26.778715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-58f96473-b629-41d3-b30c-f57650d2bdc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '046dcf4877a745e990cac3cb2c4daaaa'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:02:26.804848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:26.807785 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-58f96473-b629-41d3-b30c-f57650d2bdc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '046dcf4877a745e990cac3cb2c4daaaa'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:02:26.814282 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 749/1507] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:02:26 2022] DELETE /identity/v3/users/046dcf4877a745e990cac3cb2c4daaaa => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:02:26.819421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa213a-1963-4c4f-870c-2dcd8e345db9 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:26.819758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa213a-1963-4c4f-870c-2dcd8e345db9 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:26.820173 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aa213a-1963-4c4f-870c-2dcd8e345db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:26.841391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-c7aa213a-1963-4c4f-870c-2dcd8e345db9 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:02:26.842639 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c7aa213a-1963-4c4f-870c-2dcd8e345db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:26.853626 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:26.914024 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 759/1508] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:02:26.923841 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f96473-b629-41d3-b30c-f57650d2bdc8 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:02:26.925466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-58f96473-b629-41d3-b30c-f57650d2bdc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:27.015465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f96473-b629-41d3-b30c-f57650d2bdc8 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:02:27.020559 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb0f02-3b67-4887-b8c7-69c737e78b5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:02:27.021222 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb0f02-3b67-4887-b8c7-69c737e78b5a service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:27.021479 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb0f02-3b67-4887-b8c7-69c737e78b5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:27.021755 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb0f02-3b67-4887-b8c7-69c737e78b5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:27.032578 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb0f02-3b67-4887-b8c7-69c737e78b5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:02:27.034096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efb0f02-3b67-4887-b8c7-69c737e78b5a service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:02:27.046631 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 750/1509] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:02:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:02:27.734914 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa213a-1963-4c4f-870c-2dcd8e345db9 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:02:27.736933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c7aa213a-1963-4c4f-870c-2dcd8e345db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:02:27.750280 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aa213a-1963-4c4f-870c-2dcd8e345db9 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:02:27.755866 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b17029a-9bf1-42ec-8451-fe32e3f4aa04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:02:27.756763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17029a-9bf1-42ec-8451-fe32e3f4aa04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:02:27.757057 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17029a-9bf1-42ec-8451-fe32e3f4aa04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:02:27.757270 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b17029a-9bf1-42ec-8451-fe32e3f4aa04 admin admin] PATH_INFO: `/v3/projects/3d800782b5cd44c2819368dd44fe6cf3` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:02:27.758851 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b17029a-9bf1-42ec-8451-fe32e3f4aa04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d800782b5cd44c2819368dd44fe6cf3)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:02:27.760683 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: WARNING py.warnings [None req-3b17029a-9bf1-42ec-8451-fe32e3f4aa04 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:02:27.760683 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: warnings.warn(msg) Aug 02 09:02:27.760683 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: Aug 02 09:02:27.761119 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b17029a-9bf1-42ec-8451-fe32e3f4aa04 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:02:27.928197 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-3b17029a-9bf1-42ec-8451-fe32e3f4aa04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d800782b5cd44c2819368dd44fe6cf3', 'request_id': 'req-3b17029a-9bf1-42ec-8451-fe32e3f4aa04'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:02:27.943010 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-3b17029a-9bf1-42ec-8451-fe32e3f4aa04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d800782b5cd44c2819368dd44fe6cf3', 'request_id': 'req-3b17029a-9bf1-42ec-8451-fe32e3f4aa04'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:02:28.029808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-3b17029a-9bf1-42ec-8451-fe32e3f4aa04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d800782b5cd44c2819368dd44fe6cf3', 'request_id': 'req-3b17029a-9bf1-42ec-8451-fe32e3f4aa04'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:02:28.042825 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-3b17029a-9bf1-42ec-8451-fe32e3f4aa04 admin admin] The token cache is being invalidate because project 3d800782b5cd44c2819368dd44fe6cf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:02:28.043120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-3b17029a-9bf1-42ec-8451-fe32e3f4aa04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:02:28.044077 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-3b17029a-9bf1-42ec-8451-fe32e3f4aa04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:02:28.046384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 760/1510] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:02:27 2022] DELETE /identity/v3/projects/3d800782b5cd44c2819368dd44fe6cf3 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:02:37.040668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:02:38.039411 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:09.975632 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37961a-07aa-4407-ba10-310b2680eb58 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:09.976265 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37961a-07aa-4407-ba10-310b2680eb58 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:09.976265 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e37961a-07aa-4407-ba10-310b2680eb58 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:09.977577 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:04:09.982620 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70dbbd-1b8d-45bb-9da3-5f83a4ffb8c9 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:09.983059 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70dbbd-1b8d-45bb-9da3-5f83a4ffb8c9 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:09.983175 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70dbbd-1b8d-45bb-9da3-5f83a4ffb8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:10.002630 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-eb70dbbd-1b8d-45bb-9da3-5f83a4ffb8c9 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:10.004667 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-eb70dbbd-1b8d-45bb-9da3-5f83a4ffb8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:10.063742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:04:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:10.218471 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37961a-07aa-4407-ba10-310b2680eb58 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:10.224034 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4e37961a-07aa-4407-ba10-310b2680eb58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:10.326277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e37961a-07aa-4407-ba10-310b2680eb58 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:10.332615 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ffddf2-2280-4b82-a786-9e0642d9bfd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:10.332955 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffddf2-2280-4b82-a786-9e0642d9bfd0 service nova] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:10.333242 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffddf2-2280-4b82-a786-9e0642d9bfd0 service nova] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:10.333712 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ffddf2-2280-4b82-a786-9e0642d9bfd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:10.348774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ffddf2-2280-4b82-a786-9e0642d9bfd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:10.350408 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ffddf2-2280-4b82-a786-9e0642d9bfd0 service nova] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:10.360802 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 752/1513] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:04:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:11.196000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca136a84-801c-4a85-bf41-531236ed5add None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:11.196567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca136a84-801c-4a85-bf41-531236ed5add None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:11.196567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca136a84-801c-4a85-bf41-531236ed5add None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:11.198061 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:04:11.202896 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592129f8-1732-4873-886d-ecd549155344 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:11.203567 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592129f8-1732-4873-886d-ecd549155344 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:11.203708 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592129f8-1732-4873-886d-ecd549155344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:11.223776 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-592129f8-1732-4873-886d-ecd549155344 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:11.225030 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-592129f8-1732-4873-886d-ecd549155344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:11.233528 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:11.388909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca136a84-801c-4a85-bf41-531236ed5add None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:11.403755 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca136a84-801c-4a85-bf41-531236ed5add None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:11.409184 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e422d9c-6230-4f95-b05d-ba574460026f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:11.409789 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e422d9c-6230-4f95-b05d-ba574460026f service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:11.410103 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e422d9c-6230-4f95-b05d-ba574460026f service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:11.410380 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e422d9c-6230-4f95-b05d-ba574460026f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:11.418548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e422d9c-6230-4f95-b05d-ba574460026f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:11.420130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e422d9c-6230-4f95-b05d-ba574460026f service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:11.429860 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 763/1516] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:04:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:12.042543 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d5ad0-bfa2-4c47-a5c1-d10d57dbe5fc None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:12.042826 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d5ad0-bfa2-4c47-a5c1-d10d57dbe5fc None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:12.043116 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d5ad0-bfa2-4c47-a5c1-d10d57dbe5fc None None] PATH_INFO: `/v3` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:12.044407 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 754/1517] 10.222.0.72 () {54 vars in 925 bytes} [Tue Aug 2 09:04:12 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:04:12.050600 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5177b8e-b8e4-4225-99f6-27dc02bebb02 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:12.050897 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5177b8e-b8e4-4225-99f6-27dc02bebb02 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:12.051181 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5177b8e-b8e4-4225-99f6-27dc02bebb02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:12.101400 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-e5177b8e-b8e4-4225-99f6-27dc02bebb02 None None] MFA Rules not processed for user `cad5cc9927374836b0cc74db85b686fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:12.102641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e5177b8e-b8e4-4225-99f6-27dc02bebb02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:12.177410 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 764/1518] 10.222.0.72 () {58 vars in 1015 bytes} [Tue Aug 2 09:04:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:21.218871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=49 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:21.279223 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9319028-c002-42fb-90b4-d725759011fa None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:21.279491 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9319028-c002-42fb-90b4-d725759011fa None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:21.279772 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9319028-c002-42fb-90b4-d725759011fa None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:21.281127 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 755/1519] 10.222.0.46 () {54 vars in 961 bytes} [Tue Aug 2 09:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:04:21.286375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb6c67-dbed-428e-bcea-f026ae6e3c55 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:21.286881 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb6c67-dbed-428e-bcea-f026ae6e3c55 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:21.286982 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb6c67-dbed-428e-bcea-f026ae6e3c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:21.308599 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-b9cb6c67-dbed-428e-bcea-f026ae6e3c55 None None] MFA Rules not processed for user `cad5cc9927374836b0cc74db85b686fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:21.309811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b9cb6c67-dbed-428e-bcea-f026ae6e3c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:21.318831 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 765/1520] 10.222.0.46 () {58 vars in 1058 bytes} [Tue Aug 2 09:04:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:21.327128 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9319028-c002-42fb-90b4-d725759011fa None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:21.341395 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9319028-c002-42fb-90b4-d725759011fa None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:21.346645 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e1a578-0148-490e-a9fb-ab1de8b4cb1e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad5cc9927374836b0cc74db85b686fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:21.347248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1a578-0148-490e-a9fb-ab1de8b4cb1e service placement] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:21.347559 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1a578-0148-490e-a9fb-ab1de8b4cb1e service placement] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:21.347835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1a578-0148-490e-a9fb-ab1de8b4cb1e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:21.356698 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e1a578-0148-490e-a9fb-ab1de8b4cb1e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:21.358329 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e1a578-0148-490e-a9fb-ab1de8b4cb1e service placement] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:21.368163 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 756/1521] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:04:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:22.686594 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29736c-cd03-48ad-87e2-410ddc42b26f None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:22.687150 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29736c-cd03-48ad-87e2-410ddc42b26f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:22.687259 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f29736c-cd03-48ad-87e2-410ddc42b26f None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:22.688607 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:04:22.693639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdebabe-7aa4-4d03-9f0c-ccff8478823c None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:22.694117 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdebabe-7aa4-4d03-9f0c-ccff8478823c None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:22.694242 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdebabe-7aa4-4d03-9f0c-ccff8478823c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:22.715670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-8cdebabe-7aa4-4d03-9f0c-ccff8478823c None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:22.717276 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-8cdebabe-7aa4-4d03-9f0c-ccff8478823c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:22.726542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:22.880883 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29736c-cd03-48ad-87e2-410ddc42b26f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:22.902409 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f29736c-cd03-48ad-87e2-410ddc42b26f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:22.909042 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f80bf3-aa3c-4228-96e5-72464d5a8996 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:22.919029 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f80bf3-aa3c-4228-96e5-72464d5a8996 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:22.919029 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f80bf3-aa3c-4228-96e5-72464d5a8996 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:22.919029 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f80bf3-aa3c-4228-96e5-72464d5a8996 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:22.919342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f80bf3-aa3c-4228-96e5-72464d5a8996 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:22.920883 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f80bf3-aa3c-4228-96e5-72464d5a8996 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:22.932057 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 767/1524] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:04:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:23.769676 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af670c-f945-4d51-b3e4-e82a0203b6c7 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:23.770417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af670c-f945-4d51-b3e4-e82a0203b6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:23.770417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28af670c-f945-4d51-b3e4-e82a0203b6c7 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:23.771646 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 09:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:04:23.777412 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432cf96-fafb-43ef-a8bf-02e53bfc1e69 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:23.777936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432cf96-fafb-43ef-a8bf-02e53bfc1e69 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:23.778057 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432cf96-fafb-43ef-a8bf-02e53bfc1e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:23.799166 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-d432cf96-fafb-43ef-a8bf-02e53bfc1e69 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:23.800391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d432cf96-fafb-43ef-a8bf-02e53bfc1e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:23.808551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 09:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:23.965591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af670c-f945-4d51-b3e4-e82a0203b6c7 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:23.982351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28af670c-f945-4d51-b3e4-e82a0203b6c7 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:23.988011 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2dc5de-7327-4626-996d-b7ab4be285b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:23.988632 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2dc5de-7327-4626-996d-b7ab4be285b3 service nova] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:23.988915 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2dc5de-7327-4626-996d-b7ab4be285b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:23.989196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2dc5de-7327-4626-996d-b7ab4be285b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:23.999444 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2dc5de-7327-4626-996d-b7ab4be285b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:24.001042 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2dc5de-7327-4626-996d-b7ab4be285b3 service nova] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:24.011233 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 759/1527] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:04:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:27.957226 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79858963-aaac-470c-934b-eed1b3034f58 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:27.957765 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79858963-aaac-470c-934b-eed1b3034f58 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:27.957765 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79858963-aaac-470c-934b-eed1b3034f58 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:27.959325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:04:27.965046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884be29b-5f7d-491a-8a9d-130b1d002eef None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:27.965515 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884be29b-5f7d-491a-8a9d-130b1d002eef None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:27.965611 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884be29b-5f7d-491a-8a9d-130b1d002eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:27.985360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-884be29b-5f7d-491a-8a9d-130b1d002eef None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:27.986585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-884be29b-5f7d-491a-8a9d-130b1d002eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:27.994438 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:04:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:28.073813 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79858963-aaac-470c-934b-eed1b3034f58 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:28.075540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-79858963-aaac-470c-934b-eed1b3034f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:28.105452 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:28.136570 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79858963-aaac-470c-934b-eed1b3034f58 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:28.140455 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ae946b-a0de-4c24-9d1a-d6cc140d5095 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:28.141076 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae946b-a0de-4c24-9d1a-d6cc140d5095 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:28.141406 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae946b-a0de-4c24-9d1a-d6cc140d5095 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:28.141681 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ae946b-a0de-4c24-9d1a-d6cc140d5095 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:28.152564 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae946b-a0de-4c24-9d1a-d6cc140d5095 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:28.154316 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ae946b-a0de-4c24-9d1a-d6cc140d5095 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:28.164761 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 770/1530] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:04:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:30.200273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc56a7f-8caa-4d9c-9dbf-d3f5476f6b1b None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:30.200841 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc56a7f-8caa-4d9c-9dbf-d3f5476f6b1b None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:30.201111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc56a7f-8caa-4d9c-9dbf-d3f5476f6b1b None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:30.203708 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:04:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:04:30.212064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692e587-57de-48b6-ab23-8f92f2847d92 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:30.213061 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692e587-57de-48b6-ab23-8f92f2847d92 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:30.213167 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692e587-57de-48b6-ab23-8f92f2847d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:30.237259 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-c692e587-57de-48b6-ab23-8f92f2847d92 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:30.238544 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c692e587-57de-48b6-ab23-8f92f2847d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:30.254779 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:04:30.291849 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc56a7f-8caa-4d9c-9dbf-d3f5476f6b1b None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:30.303208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc56a7f-8caa-4d9c-9dbf-d3f5476f6b1b None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:30.307012 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac583eb-acf3-47da-808d-195e3dce2c45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:30.307632 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac583eb-acf3-47da-808d-195e3dce2c45 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:30.307923 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac583eb-acf3-47da-808d-195e3dce2c45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:30.308197 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac583eb-acf3-47da-808d-195e3dce2c45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:30.319263 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac583eb-acf3-47da-808d-195e3dce2c45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:30.320801 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac583eb-acf3-47da-808d-195e3dce2c45 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:30.332694 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 762/1533] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:04:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:33.824956 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb709ef-e2a5-4672-92a2-e535864e5b7b None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:33.825585 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb709ef-e2a5-4672-92a2-e535864e5b7b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:33.825585 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb709ef-e2a5-4672-92a2-e535864e5b7b None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:33.826917 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:04:33.832590 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b113e7-665d-48c5-88db-58580d8a3888 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:33.833084 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b113e7-665d-48c5-88db-58580d8a3888 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:33.833303 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b113e7-665d-48c5-88db-58580d8a3888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:33.853201 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-e6b113e7-665d-48c5-88db-58580d8a3888 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:33.854850 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e6b113e7-665d-48c5-88db-58580d8a3888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:33.862882 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:34.178161 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb709ef-e2a5-4672-92a2-e535864e5b7b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:34.179883 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2eb709ef-e2a5-4672-92a2-e535864e5b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:34.237819 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb709ef-e2a5-4672-92a2-e535864e5b7b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:34.241535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90271e3-bcc9-4589-9fa0-c2956eadfa0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:34.242178 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90271e3-bcc9-4589-9fa0-c2956eadfa0f service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:34.242462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90271e3-bcc9-4589-9fa0-c2956eadfa0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:34.242741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90271e3-bcc9-4589-9fa0-c2956eadfa0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:34.252248 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90271e3-bcc9-4589-9fa0-c2956eadfa0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:34.253815 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90271e3-bcc9-4589-9fa0-c2956eadfa0f service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:34.263765 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 773/1536] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:04:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:04:36.310822 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d1108-e331-42c7-befe-afe5c072acb7 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:36.311337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d1108-e331-42c7-befe-afe5c072acb7 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:36.311337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d1108-e331-42c7-befe-afe5c072acb7 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:36.312674 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 09:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:04:36.318197 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034c02d-f6f8-4a29-bd1a-2f0348b8d434 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:36.318663 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034c02d-f6f8-4a29-bd1a-2f0348b8d434 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:36.319091 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4034c02d-f6f8-4a29-bd1a-2f0348b8d434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:36.340708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-4034c02d-f6f8-4a29-bd1a-2f0348b8d434 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:04:36.341991 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4034c02d-f6f8-4a29-bd1a-2f0348b8d434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:36.350459 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 09:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:04:36.661420 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d1108-e331-42c7-befe-afe5c072acb7 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:04:36.663279 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-256d1108-e331-42c7-befe-afe5c072acb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:04:36.679377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d1108-e331-42c7-befe-afe5c072acb7 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:04:36.686061 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b85be9-4320-4116-9ae9-0e966905f8ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:04:36.686699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b85be9-4320-4116-9ae9-0e966905f8ce service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:04:36.686982 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b85be9-4320-4116-9ae9-0e966905f8ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:04:36.687263 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b85be9-4320-4116-9ae9-0e966905f8ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:04:36.698733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b85be9-4320-4116-9ae9-0e966905f8ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:04:36.702483 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b85be9-4320-4116-9ae9-0e966905f8ce service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:04:36.711949 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 765/1539] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:04:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:04:46.334466 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=47 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:04:46.706585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=33 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:10.255893 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d0b20-4c1b-432c-a7f2-fa90d69ec245 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:10.257564 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d0b20-4c1b-432c-a7f2-fa90d69ec245 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:10.257564 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d0b20-4c1b-432c-a7f2-fa90d69ec245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:10.286983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-194d0b20-4c1b-432c-a7f2-fa90d69ec245 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:10.288216 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-194d0b20-4c1b-432c-a7f2-fa90d69ec245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:10.301007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 775/1540] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:10.310536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9660c3d-03d0-4c16-a857-ea958a035ca1 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:10.311124 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9660c3d-03d0-4c16-a857-ea958a035ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:10.311124 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9660c3d-03d0-4c16-a857-ea958a035ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:10.328551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa2b42-b6bf-43b4-82d1-b14389c1839a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:10.328870 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa2b42-b6bf-43b4-82d1-b14389c1839a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:10.329374 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fa2b42-b6bf-43b4-82d1-b14389c1839a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:10.339320 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-a9660c3d-03d0-4c16-a857-ea958a035ca1 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:10.340776 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-a9660c3d-03d0-4c16-a857-ea958a035ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:10.357024 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 766/1541] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:10.361602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9660c3d-03d0-4c16-a857-ea958a035ca1 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:10.366985 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f6fa2b42-b6bf-43b4-82d1-b14389c1839a None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:10.368226 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f6fa2b42-b6bf-43b4-82d1-b14389c1839a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:10.375470 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9660c3d-03d0-4c16-a857-ea958a035ca1 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:10.382078 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e4361-bf0d-45a9-a381-11be05dfd3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:10.382711 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e4361-bf0d-45a9-a381-11be05dfd3d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:10.383130 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e4361-bf0d-45a9-a381-11be05dfd3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:10.383409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e4361-bf0d-45a9-a381-11be05dfd3d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:10.384118 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e4361-bf0d-45a9-a381-11be05dfd3d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:06:10.384427 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e4361-bf0d-45a9-a381-11be05dfd3d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:10.385451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: WARNING py.warnings [None req-9e9e4361-bf0d-45a9-a381-11be05dfd3d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:06:10.385451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: warnings.warn(msg) Aug 02 09:06:10.385451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: Aug 02 09:06:10.387557 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9e4361-bf0d-45a9-a381-11be05dfd3d1 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:10.389260 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 776/1542] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:06:10.399062 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092a7e7-1560-489c-8abf-e42b49212609 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:10.399391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092a7e7-1560-489c-8abf-e42b49212609 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:10.399617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a092a7e7-1560-489c-8abf-e42b49212609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:10.415419 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 767/1543] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:06:10.425287 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e4361-bf0d-45a9-a381-11be05dfd3d1 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:10.439709 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e4361-bf0d-45a9-a381-11be05dfd3d1 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:10.450354 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d66cdd-5d6e-4b73-afaa-20d719cc9494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:10.450945 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-a092a7e7-1560-489c-8abf-e42b49212609 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:10.451081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d66cdd-5d6e-4b73-afaa-20d719cc9494 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:10.451254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d66cdd-5d6e-4b73-afaa-20d719cc9494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:10.451528 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d66cdd-5d6e-4b73-afaa-20d719cc9494 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:10.452207 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d66cdd-5d6e-4b73-afaa-20d719cc9494 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:10.453215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a092a7e7-1560-489c-8abf-e42b49212609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:10.454580 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d66cdd-5d6e-4b73-afaa-20d719cc9494 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:10.487729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 777/1544] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:06:10.492343 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092a7e7-1560-489c-8abf-e42b49212609 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:10.510322 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a092a7e7-1560-489c-8abf-e42b49212609 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:10.518859 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a77ae-e125-499f-b9c9-c89b623df11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:10.519534 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a77ae-e125-499f-b9c9-c89b623df11b admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:10.519819 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a77ae-e125-499f-b9c9-c89b623df11b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:10.520103 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a77ae-e125-499f-b9c9-c89b623df11b admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:10.520825 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a77ae-e125-499f-b9c9-c89b623df11b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:06:10.521137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a77ae-e125-499f-b9c9-c89b623df11b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:10.523030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a77ae-e125-499f-b9c9-c89b623df11b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:10.541772 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 778/1545] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:06:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:06:10.551025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a77ae-e125-499f-b9c9-c89b623df11b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:10.562514 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a77ae-e125-499f-b9c9-c89b623df11b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:10.575109 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d1d750-e5db-49df-8e06-74c357355afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:10.580543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1d750-e5db-49df-8e06-74c357355afb admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:10.580543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1d750-e5db-49df-8e06-74c357355afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:10.580543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d1d750-e5db-49df-8e06-74c357355afb admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:10.580543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1d750-e5db-49df-8e06-74c357355afb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:10.581147 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 768/1546] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:06:10 2022] POST /identity/v3/projects => generated 389 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:06:10.581304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: WARNING py.warnings [None req-26d1d750-e5db-49df-8e06-74c357355afb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:06:10.581304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: warnings.warn(msg) Aug 02 09:06:10.581304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: Aug 02 09:06:10.581304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d1d750-e5db-49df-8e06-74c357355afb admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:10.586390 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d66cdd-5d6e-4b73-afaa-20d719cc9494 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:10.730825 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d66cdd-5d6e-4b73-afaa-20d719cc9494 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:10.734539 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 779/1547] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:06:10 2022] POST /identity/v3/projects => generated 383 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:06:10.735930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c09b8-3bc9-469c-9a34-5d98333f1b7e None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:10.736215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c09b8-3bc9-469c-9a34-5d98333f1b7e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:10.736498 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c09b8-3bc9-469c-9a34-5d98333f1b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:10.739806 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58503984-1e40-4158-8855-215803e341da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:10.740860 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58503984-1e40-4158-8855-215803e341da admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:10.741368 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58503984-1e40-4158-8855-215803e341da admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:10.741768 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58503984-1e40-4158-8855-215803e341da admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:10.742378 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58503984-1e40-4158-8855-215803e341da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:10.743433 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: WARNING py.warnings [None req-58503984-1e40-4158-8855-215803e341da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:06:10.743433 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: warnings.warn(msg) Aug 02 09:06:10.743433 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: Aug 02 09:06:10.743433 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58503984-1e40-4158-8855-215803e341da admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:10.766766 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-016c09b8-3bc9-469c-9a34-5d98333f1b7e None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:10.767675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-016c09b8-3bc9-469c-9a34-5d98333f1b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:10.837836 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 769/1548] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:06:10 2022] POST /identity/v3/users => generated 295 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:06:10.839683 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9422b-3a83-49e4-ad20-681b198792f5 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:10.840171 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9422b-3a83-49e4-ad20-681b198792f5 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:10.840633 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9422b-3a83-49e4-ad20-681b198792f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:10.848217 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 780/1549] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:10.855848 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372bdf3-a9e1-4fe7-addd-1ec0cc5c4b7f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:10.856139 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372bdf3-a9e1-4fe7-addd-1ec0cc5c4b7f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:10.856420 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372bdf3-a9e1-4fe7-addd-1ec0cc5c4b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:10.867700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-43c9422b-3a83-49e4-ad20-681b198792f5 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:10.867700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-43c9422b-3a83-49e4-ad20-681b198792f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:10.874852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 770/1550] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:10.878604 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027d4ce-d68f-4625-83aa-f8692d71d70f None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:10.878909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027d4ce-d68f-4625-83aa-f8692d71d70f None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:10.879172 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027d4ce-d68f-4625-83aa-f8692d71d70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:10.891721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-4372bdf3-a9e1-4fe7-addd-1ec0cc5c4b7f None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:10.892957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4372bdf3-a9e1-4fe7-addd-1ec0cc5c4b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:10.905779 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-b027d4ce-d68f-4625-83aa-f8692d71d70f None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:10.907059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-b027d4ce-d68f-4625-83aa-f8692d71d70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:10.912106 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 781/1551] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:10.914620 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca8518-b13e-4e95-881c-3e4d66a3f7fa None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:10.915128 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca8518-b13e-4e95-881c-3e4d66a3f7fa None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:10.915654 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca8518-b13e-4e95-881c-3e4d66a3f7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:10.920337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 771/1552] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:10.924434 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027d4ce-d68f-4625-83aa-f8692d71d70f None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:10.936168 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b027d4ce-d68f-4625-83aa-f8692d71d70f None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:10.941670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70e3cc-5fe5-4572-822c-ab22d8608ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:10.942611 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70e3cc-5fe5-4572-822c-ab22d8608ab2 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:10.942945 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70e3cc-5fe5-4572-822c-ab22d8608ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:10.943380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70e3cc-5fe5-4572-822c-ab22d8608ab2 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:10.944205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be70e3cc-5fe5-4572-822c-ab22d8608ab2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:10.947252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-7dca8518-b13e-4e95-881c-3e4d66a3f7fa None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:10.947582 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be70e3cc-5fe5-4572-822c-ab22d8608ab2 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:10.949064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7dca8518-b13e-4e95-881c-3e4d66a3f7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:10.960659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 782/1553] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:10.962130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cd872-7fe1-4cc2-b7ca-943586b1f6f1 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:10.962456 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cd872-7fe1-4cc2-b7ca-943586b1f6f1 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:10.963156 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3cd872-7fe1-4cc2-b7ca-943586b1f6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:10.984171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-4a3cd872-7fe1-4cc2-b7ca-943586b1f6f1 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:10.985408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4a3cd872-7fe1-4cc2-b7ca-943586b1f6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:10.994018 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 783/1554] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:10.996546 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cd872-7fe1-4cc2-b7ca-943586b1f6f1 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:11.006069 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3cd872-7fe1-4cc2-b7ca-943586b1f6f1 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:11.010803 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f7614-121d-4856-b757-67763b1e42e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:11.012069 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f7614-121d-4856-b757-67763b1e42e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:11.012358 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f7614-121d-4856-b757-67763b1e42e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:11.012638 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f7614-121d-4856-b757-67763b1e42e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:11.013336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f7614-121d-4856-b757-67763b1e42e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:11.014997 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f7614-121d-4856-b757-67763b1e42e6 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:11.026211 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 784/1555] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:11.027511 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c13a16-a048-47d7-864f-e30f9e81d7a6 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:11.027801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c13a16-a048-47d7-864f-e30f9e81d7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:11.028083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c13a16-a048-47d7-864f-e30f9e81d7a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:11.054113 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-74c13a16-a048-47d7-864f-e30f9e81d7a6 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:11.055565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-74c13a16-a048-47d7-864f-e30f9e81d7a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:11.064499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 785/1556] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:11.065874 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494e481-6372-4b9c-943a-f7f8a531067e None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:11.066130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494e481-6372-4b9c-943a-f7f8a531067e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:11.066573 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494e481-6372-4b9c-943a-f7f8a531067e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:11.077002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 772/1557] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:06:10 2022] POST /identity/v3/users => generated 292 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:11.079610 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8c271-f101-4355-931e-0e871480729b None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:11.079610 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8c271-f101-4355-931e-0e871480729b None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:11.079610 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8c271-f101-4355-931e-0e871480729b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:11.087556 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-7494e481-6372-4b9c-943a-f7f8a531067e None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:11.088808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7494e481-6372-4b9c-943a-f7f8a531067e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:11.102127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 786/1558] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:11.102127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541fb3b-19ab-4fbe-81c2-3b5b0867284a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:11.102127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541fb3b-19ab-4fbe-81c2-3b5b0867284a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:11.102127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e541fb3b-19ab-4fbe-81c2-3b5b0867284a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:11.104188 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-7ed8c271-f101-4355-931e-0e871480729b None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:11.105428 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-7ed8c271-f101-4355-931e-0e871480729b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:11.113748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 773/1559] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:11.115075 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845f80d-f737-4c92-a891-ca05c9ecba91 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:11.115362 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845f80d-f737-4c92-a891-ca05c9ecba91 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:11.115639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845f80d-f737-4c92-a891-ca05c9ecba91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:11.120199 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-e541fb3b-19ab-4fbe-81c2-3b5b0867284a None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:11.122031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e541fb3b-19ab-4fbe-81c2-3b5b0867284a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:11.130896 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 787/1560] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:11.131875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf93a8-baab-493b-b2ac-e7d64fe5ed24 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:11.132170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf93a8-baab-493b-b2ac-e7d64fe5ed24 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:11.132472 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf93a8-baab-493b-b2ac-e7d64fe5ed24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:11.137294 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-8845f80d-f737-4c92-a891-ca05c9ecba91 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:11.138567 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-8845f80d-f737-4c92-a891-ca05c9ecba91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:11.147662 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 774/1561] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:11.148576 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845f80d-f737-4c92-a891-ca05c9ecba91 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:11.154918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-d8cf93a8-baab-493b-b2ac-e7d64fe5ed24 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:11.156259 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d8cf93a8-baab-493b-b2ac-e7d64fe5ed24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:11.160890 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845f80d-f737-4c92-a891-ca05c9ecba91 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:11.168585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7077d6f-c525-492c-b9fe-1cfb174ac79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:11.173245 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 788/1562] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:11.173394 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7077d6f-c525-492c-b9fe-1cfb174ac79f admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:11.173394 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7077d6f-c525-492c-b9fe-1cfb174ac79f admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:11.173394 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7077d6f-c525-492c-b9fe-1cfb174ac79f admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:11.173668 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf93a8-baab-493b-b2ac-e7d64fe5ed24 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:11.173777 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7077d6f-c525-492c-b9fe-1cfb174ac79f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:11.174611 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7077d6f-c525-492c-b9fe-1cfb174ac79f admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:11.186038 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf93a8-baab-493b-b2ac-e7d64fe5ed24 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:11.195139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 775/1563] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:11.195361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ea7af-52be-4f80-8f88-b9689ee53cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:11.195490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7077d6f-c525-492c-b9fe-1cfb174ac79f admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:11.195601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ea7af-52be-4f80-8f88-b9689ee53cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:11.195601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ea7af-52be-4f80-8f88-b9689ee53cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:11.195601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ea7af-52be-4f80-8f88-b9689ee53cf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:11.196250 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ea7af-52be-4f80-8f88-b9689ee53cf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:06:11.196775 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ea7af-52be-4f80-8f88-b9689ee53cf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:11.200250 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ea7af-52be-4f80-8f88-b9689ee53cf0 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:11.210817 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7077d6f-c525-492c-b9fe-1cfb174ac79f admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:11.218274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5bf2f-5716-494c-add5-53761f350350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:11.219235 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5bf2f-5716-494c-add5-53761f350350 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:11.219354 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 789/1564] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:06:11.219462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5bf2f-5716-494c-add5-53761f350350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:11.219824 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb5bf2f-5716-494c-add5-53761f350350 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:11.220137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ea7af-52be-4f80-8f88-b9689ee53cf0 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:11.221139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb5bf2f-5716-494c-add5-53761f350350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:11.224194 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb5bf2f-5716-494c-add5-53761f350350 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:11.231662 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ea7af-52be-4f80-8f88-b9689ee53cf0 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:11.236850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ed382-4177-4927-9c5f-50d63a9faded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:11.237465 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ed382-4177-4927-9c5f-50d63a9faded admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:11.237745 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ed382-4177-4927-9c5f-50d63a9faded admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:11.238068 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ed382-4177-4927-9c5f-50d63a9faded admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:11.238636 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 776/1565] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:11.238822 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ed382-4177-4927-9c5f-50d63a9faded admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:06:11.239061 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ed382-4177-4927-9c5f-50d63a9faded admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:11.239729 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5bf2f-5716-494c-add5-53761f350350 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:11.240868 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ed382-4177-4927-9c5f-50d63a9faded admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:11.251386 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb5bf2f-5716-494c-add5-53761f350350 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:11.255109 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 790/1566] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:06:11.255788 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ed382-4177-4927-9c5f-50d63a9faded admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:11.257083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8d121-608e-4c32-a0ec-485e4ad3c685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:11.257702 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8d121-608e-4c32-a0ec-485e4ad3c685 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:11.258009 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8d121-608e-4c32-a0ec-485e4ad3c685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:11.258288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8d121-608e-4c32-a0ec-485e4ad3c685 admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:11.259012 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8d121-608e-4c32-a0ec-485e4ad3c685 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:06:11.259313 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8d121-608e-4c32-a0ec-485e4ad3c685 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:11.262185 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f8d121-608e-4c32-a0ec-485e4ad3c685 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:11.266579 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ed382-4177-4927-9c5f-50d63a9faded admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:11.271221 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884abe0-c3f2-4e16-9197-3c8da96cb70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:11.271840 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884abe0-c3f2-4e16-9197-3c8da96cb70b admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:11.272129 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884abe0-c3f2-4e16-9197-3c8da96cb70b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:11.272416 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c884abe0-c3f2-4e16-9197-3c8da96cb70b admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:11.273098 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884abe0-c3f2-4e16-9197-3c8da96cb70b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:06:11.273403 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884abe0-c3f2-4e16-9197-3c8da96cb70b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:11.275039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c884abe0-c3f2-4e16-9197-3c8da96cb70b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:11.275587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 777/1567] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:06:11.276472 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8d121-608e-4c32-a0ec-485e4ad3c685 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:11.288270 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 791/1568] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:06:11.288479 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8d121-608e-4c32-a0ec-485e4ad3c685 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:11.289282 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884abe0-c3f2-4e16-9197-3c8da96cb70b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:11.296276 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40617392-1ee4-4760-9bfe-32e8e20ff4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:11.296897 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40617392-1ee4-4760-9bfe-32e8e20ff4f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:11.297174 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40617392-1ee4-4760-9bfe-32e8e20ff4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:11.297445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40617392-1ee4-4760-9bfe-32e8e20ff4f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:11.298381 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40617392-1ee4-4760-9bfe-32e8e20ff4f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:06:11.298740 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40617392-1ee4-4760-9bfe-32e8e20ff4f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:11.300868 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40617392-1ee4-4760-9bfe-32e8e20ff4f5 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:11.302023 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c884abe0-c3f2-4e16-9197-3c8da96cb70b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:11.306713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661a39e-c2dd-4b59-a3d1-950902eae0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:11.307332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661a39e-c2dd-4b59-a3d1-950902eae0bd admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:11.307613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661a39e-c2dd-4b59-a3d1-950902eae0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:11.307888 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661a39e-c2dd-4b59-a3d1-950902eae0bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:11.308623 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4661a39e-c2dd-4b59-a3d1-950902eae0bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:06:11.308926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4661a39e-c2dd-4b59-a3d1-950902eae0bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:11.310559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4661a39e-c2dd-4b59-a3d1-950902eae0bd admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:11.313063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:11.315233 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 778/1569] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:06:11.316045 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40617392-1ee4-4760-9bfe-32e8e20ff4f5 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:11.323832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 792/1570] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:06:11.331299 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661a39e-c2dd-4b59-a3d1-950902eae0bd admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:11.331541 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40617392-1ee4-4760-9bfe-32e8e20ff4f5 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:11.334384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e479a1-5027-4b14-a3bc-fc007b2d60dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:11.335346 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e479a1-5027-4b14-a3bc-fc007b2d60dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:11.335675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e479a1-5027-4b14-a3bc-fc007b2d60dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:11.335962 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e479a1-5027-4b14-a3bc-fc007b2d60dc admin admin] PATH_INFO: `/v3/projects/7e47ffad695c4b049ca730c8cc43b9f9/users/50918811a90a444eaf123b2aeb4a75b3/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:11.342541 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661a39e-c2dd-4b59-a3d1-950902eae0bd admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:11.347277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab8a76-1952-4246-a04e-aea50648c3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:11.347902 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab8a76-1952-4246-a04e-aea50648c3ce admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:11.348181 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab8a76-1952-4246-a04e-aea50648c3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:11.348459 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ab8a76-1952-4246-a04e-aea50648c3ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:11.349380 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab8a76-1952-4246-a04e-aea50648c3ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:11.349517 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:11.351067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ab8a76-1952-4246-a04e-aea50648c3ce admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:11.352737 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e479a1-5027-4b14-a3bc-fc007b2d60dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e47ffad695c4b049ca730c8cc43b9f9, user_id=50918811a90a444eaf123b2aeb4a75b3, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:11.355517 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e479a1-5027-4b14-a3bc-fc007b2d60dc admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:11.485356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 779/1571] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:11 2022] PUT /identity/v3/projects/7e47ffad695c4b049ca730c8cc43b9f9/users/50918811a90a444eaf123b2aeb4a75b3/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:11.489179 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e479a1-5027-4b14-a3bc-fc007b2d60dc admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:11.509748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 793/1572] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:06:11 2022] POST /identity/v3/projects => generated 391 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:11.510645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab8a76-1952-4246-a04e-aea50648c3ce admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:11.563266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e479a1-5027-4b14-a3bc-fc007b2d60dc admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:11.585073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ab8a76-1952-4246-a04e-aea50648c3ce admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:11.590130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278097d9-f877-4a74-901a-20be1909b71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:11.590854 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278097d9-f877-4a74-901a-20be1909b71d admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:11.591135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278097d9-f877-4a74-901a-20be1909b71d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:11.591409 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278097d9-f877-4a74-901a-20be1909b71d admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:11.592081 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278097d9-f877-4a74-901a-20be1909b71d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:11.594110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278097d9-f877-4a74-901a-20be1909b71d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:11.620455 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d96805-5630-4e25-a114-18bd2361484b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:11.621597 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d96805-5630-4e25-a114-18bd2361484b admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:11.621877 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d96805-5630-4e25-a114-18bd2361484b admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:11.622189 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d96805-5630-4e25-a114-18bd2361484b admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:11.622876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d96805-5630-4e25-a114-18bd2361484b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:11.624505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d96805-5630-4e25-a114-18bd2361484b admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:11.634863 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 780/1573] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:11 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:11.635693 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d96805-5630-4e25-a114-18bd2361484b admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:11.646263 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d96805-5630-4e25-a114-18bd2361484b admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:11.650969 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f3c00-9276-4015-bc9f-e8bcb660a2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:11.652268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f3c00-9276-4015-bc9f-e8bcb660a2fd admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:11.652268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f3c00-9276-4015-bc9f-e8bcb660a2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:11.652268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f3c00-9276-4015-bc9f-e8bcb660a2fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:11.652910 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060f3c00-9276-4015-bc9f-e8bcb660a2fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:11.654849 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060f3c00-9276-4015-bc9f-e8bcb660a2fd admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:11.702635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 794/1574] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:06:11 2022] POST /identity/v3/projects => generated 365 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:11.703472 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278097d9-f877-4a74-901a-20be1909b71d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:11.758269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 781/1575] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:06:11 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:11.758269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f3c00-9276-4015-bc9f-e8bcb660a2fd admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:11.774594 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278097d9-f877-4a74-901a-20be1909b71d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:11.828791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f3c00-9276-4015-bc9f-e8bcb660a2fd admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:11.833766 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4aaeb-4a0f-421f-8ad3-ded59efd5eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:11.834310 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76578e77-b95a-4e9f-bc4d-fb9c7a4a1c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:11.834462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4aaeb-4a0f-421f-8ad3-ded59efd5eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:11.834704 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4aaeb-4a0f-421f-8ad3-ded59efd5eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:11.834927 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76578e77-b95a-4e9f-bc4d-fb9c7a4a1c65 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:11.835057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4aaeb-4a0f-421f-8ad3-ded59efd5eb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:11.835226 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76578e77-b95a-4e9f-bc4d-fb9c7a4a1c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:11.835484 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76578e77-b95a-4e9f-bc4d-fb9c7a4a1c65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:11.835655 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4aaeb-4a0f-421f-8ad3-ded59efd5eb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:11.836127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76578e77-b95a-4e9f-bc4d-fb9c7a4a1c65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:11.837629 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4aaeb-4a0f-421f-8ad3-ded59efd5eb7 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:11.838036 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76578e77-b95a-4e9f-bc4d-fb9c7a4a1c65 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:11.849961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:11.929757 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:11.941339 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 782/1576] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:06:11 2022] POST /identity/v3/projects => generated 375 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:11.941339 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4aaeb-4a0f-421f-8ad3-ded59efd5eb7 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:11.950176 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 795/1577] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:06:11 2022] POST /identity/v3/projects => generated 405 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:11.952660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacecece-de0e-4341-b7f8-28df068c2360 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:11.953161 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacecece-de0e-4341-b7f8-28df068c2360 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:11.953657 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacecece-de0e-4341-b7f8-28df068c2360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:11.974646 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-bacecece-de0e-4341-b7f8-28df068c2360 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:11.975862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bacecece-de0e-4341-b7f8-28df068c2360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:12.009600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4aaeb-4a0f-421f-8ad3-ded59efd5eb7 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:12.038932 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 796/1578] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:12.039818 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacecece-de0e-4341-b7f8-28df068c2360 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:12.050177 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacecece-de0e-4341-b7f8-28df068c2360 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:12.055008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f976756-c9a8-4f9c-9eb6-fbd7bfb37cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:12.055568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f976756-c9a8-4f9c-9eb6-fbd7bfb37cd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:12.055914 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f976756-c9a8-4f9c-9eb6-fbd7bfb37cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:12.056386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f976756-c9a8-4f9c-9eb6-fbd7bfb37cd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:12.056806 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f976756-c9a8-4f9c-9eb6-fbd7bfb37cd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:12.059502 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f976756-c9a8-4f9c-9eb6-fbd7bfb37cd7 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:12.075705 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6f126-2f0d-4648-aa20-1402102b22e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:12.076358 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6f126-2f0d-4648-aa20-1402102b22e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:12.076666 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6f126-2f0d-4648-aa20-1402102b22e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:12.076922 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6f126-2f0d-4648-aa20-1402102b22e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:12.077626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6f126-2f0d-4648-aa20-1402102b22e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:12.082930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6f126-2f0d-4648-aa20-1402102b22e9 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:12.174736 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 797/1579] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:06:12 2022] POST /identity/v3/users => generated 296 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:12.174736 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f976756-c9a8-4f9c-9eb6-fbd7bfb37cd7 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:12.184556 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f976756-c9a8-4f9c-9eb6-fbd7bfb37cd7 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:12.190787 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7b044-a5c4-467f-bb75-add341a84008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:12.191454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7b044-a5c4-467f-bb75-add341a84008 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:12.191738 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7b044-a5c4-467f-bb75-add341a84008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:12.192012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7b044-a5c4-467f-bb75-add341a84008 admin admin] PATH_INFO: `/v3/projects/b9df882a572a4bed8966779e50eb7e7c/users/806f0c03435e4c5daeadaa4eb437d936/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:12.196431 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 783/1580] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:06:11 2022] POST /identity/v3/projects => generated 379 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:12.196431 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6f126-2f0d-4648-aa20-1402102b22e9 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:12.210200 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7b044-a5c4-467f-bb75-add341a84008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9df882a572a4bed8966779e50eb7e7c, user_id=806f0c03435e4c5daeadaa4eb437d936, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:12.211913 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: WARNING py.warnings [None req-fdd7b044-a5c4-467f-bb75-add341a84008 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:06:12.211913 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: warnings.warn(msg) Aug 02 09:06:12.211913 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: Aug 02 09:06:12.215369 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7b044-a5c4-467f-bb75-add341a84008 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:12.259451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6f126-2f0d-4648-aa20-1402102b22e9 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:12.264336 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e9195-63f1-40a5-b1d0-28dab07fd6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:12.266461 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e9195-63f1-40a5-b1d0-28dab07fd6bd admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:12.266461 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e9195-63f1-40a5-b1d0-28dab07fd6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:12.266461 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e9195-63f1-40a5-b1d0-28dab07fd6bd admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:12.266461 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e9195-63f1-40a5-b1d0-28dab07fd6bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:12.268133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e9195-63f1-40a5-b1d0-28dab07fd6bd admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:12.291066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 798/1581] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:12 2022] PUT /identity/v3/projects/b9df882a572a4bed8966779e50eb7e7c/users/806f0c03435e4c5daeadaa4eb437d936/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:12.295180 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7b044-a5c4-467f-bb75-add341a84008 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:12.347886 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 784/1582] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:06:12 2022] POST /identity/v3/users => generated 283 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:12.348703 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e9195-63f1-40a5-b1d0-28dab07fd6bd admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:12.358064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7b044-a5c4-467f-bb75-add341a84008 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:12.359337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e9195-63f1-40a5-b1d0-28dab07fd6bd admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:12.364063 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921408e3-30cb-4378-9265-9959342a7c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:12.364677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921408e3-30cb-4378-9265-9959342a7c87 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:12.364958 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921408e3-30cb-4378-9265-9959342a7c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:12.365234 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921408e3-30cb-4378-9265-9959342a7c87 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:12.365914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921408e3-30cb-4378-9265-9959342a7c87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:12.367814 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921408e3-30cb-4378-9265-9959342a7c87 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:12.370791 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bcbdf0-d0f5-494d-a8e1-1733adc500e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:12.373124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcbdf0-d0f5-494d-a8e1-1733adc500e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:12.374153 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcbdf0-d0f5-494d-a8e1-1733adc500e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:12.375789 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcbdf0-d0f5-494d-a8e1-1733adc500e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:12.377338 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bcbdf0-d0f5-494d-a8e1-1733adc500e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:12.379517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bcbdf0-d0f5-494d-a8e1-1733adc500e5 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:12.449737 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 785/1583] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:06:12 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:12.450606 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921408e3-30cb-4378-9265-9959342a7c87 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:12.456848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:12.462026 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921408e3-30cb-4378-9265-9959342a7c87 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:12.467294 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96f2f7-6515-47c2-8019-dfd117607329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:12.467953 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96f2f7-6515-47c2-8019-dfd117607329 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:12.468240 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96f2f7-6515-47c2-8019-dfd117607329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:12.472381 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96f2f7-6515-47c2-8019-dfd117607329 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:12.472381 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96f2f7-6515-47c2-8019-dfd117607329 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:12.472381 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96f2f7-6515-47c2-8019-dfd117607329 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:12.473906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 799/1584] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:06:12 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:12.474828 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bcbdf0-d0f5-494d-a8e1-1733adc500e5 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:12.521161 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:12.534034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bcbdf0-d0f5-494d-a8e1-1733adc500e5 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:12.537850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1b9f2-7c44-4ddb-b75c-0ad5a9ea1e75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:12.538503 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1b9f2-7c44-4ddb-b75c-0ad5a9ea1e75 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:12.538786 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1b9f2-7c44-4ddb-b75c-0ad5a9ea1e75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:12.539083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1b9f2-7c44-4ddb-b75c-0ad5a9ea1e75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:12.549007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1b9f2-7c44-4ddb-b75c-0ad5a9ea1e75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:12.553715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1b9f2-7c44-4ddb-b75c-0ad5a9ea1e75 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:12.562242 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 800/1585] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:12.562242 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1b9f2-7c44-4ddb-b75c-0ad5a9ea1e75 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:12.567963 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 786/1586] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:06:12 2022] POST /identity/v3/users => generated 303 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:12.569579 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96f2f7-6515-47c2-8019-dfd117607329 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:12.574628 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1b9f2-7c44-4ddb-b75c-0ad5a9ea1e75 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:12.579970 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd5cd3-3e25-497a-ac84-b6c0683dd840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:12.580599 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd5cd3-3e25-497a-ac84-b6c0683dd840 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:12.580885 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd5cd3-3e25-497a-ac84-b6c0683dd840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:12.581164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd5cd3-3e25-497a-ac84-b6c0683dd840 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:12.581866 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd5cd3-3e25-497a-ac84-b6c0683dd840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:12.583669 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96f2f7-6515-47c2-8019-dfd117607329 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:12.583811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd5cd3-3e25-497a-ac84-b6c0683dd840 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:12.589103 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc53bbe-2a71-428e-a35c-b794287e21a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:12.590531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc53bbe-2a71-428e-a35c-b794287e21a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:12.590895 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc53bbe-2a71-428e-a35c-b794287e21a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:12.591258 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc53bbe-2a71-428e-a35c-b794287e21a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:12.592010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc53bbe-2a71-428e-a35c-b794287e21a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:12.594075 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc53bbe-2a71-428e-a35c-b794287e21a4 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:12.594871 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 801/1587] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:12.596149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fac208-3aa7-4cd2-9305-b9e42eefc31c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:12.596460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fac208-3aa7-4cd2-9305-b9e42eefc31c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:12.596729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fac208-3aa7-4cd2-9305-b9e42eefc31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:12.618891 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-99fac208-3aa7-4cd2-9305-b9e42eefc31c None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:12.620014 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-99fac208-3aa7-4cd2-9305-b9e42eefc31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:12.627818 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 802/1588] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:12.628637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fac208-3aa7-4cd2-9305-b9e42eefc31c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:12.638610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fac208-3aa7-4cd2-9305-b9e42eefc31c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:12.645025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37182f95-c9a0-4198-aba7-29bee623385b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:12.648657 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37182f95-c9a0-4198-aba7-29bee623385b admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:12.648657 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37182f95-c9a0-4198-aba7-29bee623385b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:12.648657 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37182f95-c9a0-4198-aba7-29bee623385b admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:12.648657 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37182f95-c9a0-4198-aba7-29bee623385b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:12.648657 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37182f95-c9a0-4198-aba7-29bee623385b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:12.659944 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 803/1589] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:12.660793 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37182f95-c9a0-4198-aba7-29bee623385b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:12.671656 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37182f95-c9a0-4198-aba7-29bee623385b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:12.676373 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a227302-6799-45a3-81b2-72577ec4917c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:12.677090 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a227302-6799-45a3-81b2-72577ec4917c admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:12.679347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a227302-6799-45a3-81b2-72577ec4917c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:12.679347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a227302-6799-45a3-81b2-72577ec4917c admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:12.679347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a227302-6799-45a3-81b2-72577ec4917c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:12.681324 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a227302-6799-45a3-81b2-72577ec4917c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:12.693103 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 804/1590] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:12.694244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a227302-6799-45a3-81b2-72577ec4917c admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:12.705633 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a227302-6799-45a3-81b2-72577ec4917c admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:12.712132 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c68b04-da9f-4242-b28c-b6de0969846a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:12.712872 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c68b04-da9f-4242-b28c-b6de0969846a admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:12.713156 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c68b04-da9f-4242-b28c-b6de0969846a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:12.713435 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c68b04-da9f-4242-b28c-b6de0969846a admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:12.714178 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c68b04-da9f-4242-b28c-b6de0969846a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:12.714373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 787/1591] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:06:12 2022] POST /identity/v3/users => generated 290 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:12.715416 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc53bbe-2a71-428e-a35c-b794287e21a4 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:12.722247 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c68b04-da9f-4242-b28c-b6de0969846a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:12.729940 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc53bbe-2a71-428e-a35c-b794287e21a4 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:12.730519 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 805/1592] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:12.730519 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c68b04-da9f-4242-b28c-b6de0969846a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:12.739915 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d04fd3-058a-4d3a-8940-b1f33676aa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:12.740616 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d04fd3-058a-4d3a-8940-b1f33676aa1e admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:12.740900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d04fd3-058a-4d3a-8940-b1f33676aa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:12.741178 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d04fd3-058a-4d3a-8940-b1f33676aa1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:12.741950 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d04fd3-058a-4d3a-8940-b1f33676aa1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:12.743529 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c68b04-da9f-4242-b28c-b6de0969846a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:12.743678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d04fd3-058a-4d3a-8940-b1f33676aa1e admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:12.754137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373467ee-b72f-40ea-888a-de1d37233d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:12.765680 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373467ee-b72f-40ea-888a-de1d37233d85 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:12.765680 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373467ee-b72f-40ea-888a-de1d37233d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:12.765680 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373467ee-b72f-40ea-888a-de1d37233d85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:12.765680 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373467ee-b72f-40ea-888a-de1d37233d85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:12.765680 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373467ee-b72f-40ea-888a-de1d37233d85 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:12.773606 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 788/1593] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:12.775184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d04fd3-058a-4d3a-8940-b1f33676aa1e admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:12.781721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 806/1594] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:12.782075 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-d3d04fd3-058a-4d3a-8940-b1f33676aa1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:12.782189 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373467ee-b72f-40ea-888a-de1d37233d85 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:12.792423 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d04fd3-058a-4d3a-8940-b1f33676aa1e admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:12.792656 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373467ee-b72f-40ea-888a-de1d37233d85 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:12.799590 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087036a4-c639-477b-9279-84acddf42f20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:12.799590 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087036a4-c639-477b-9279-84acddf42f20 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:12.799826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a798cc-c5d3-4c43-80bc-60227545bb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:12.799932 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087036a4-c639-477b-9279-84acddf42f20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:12.799932 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087036a4-c639-477b-9279-84acddf42f20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:12.800110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a798cc-c5d3-4c43-80bc-60227545bb29 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:12.800110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a798cc-c5d3-4c43-80bc-60227545bb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:12.800110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a798cc-c5d3-4c43-80bc-60227545bb29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:12.800110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a798cc-c5d3-4c43-80bc-60227545bb29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:12.800890 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a798cc-c5d3-4c43-80bc-60227545bb29 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:12.809774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087036a4-c639-477b-9279-84acddf42f20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:12.811256 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087036a4-c639-477b-9279-84acddf42f20 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:12.812132 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 807/1595] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:12.812964 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a798cc-c5d3-4c43-80bc-60227545bb29 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:12.825219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a798cc-c5d3-4c43-80bc-60227545bb29 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:12.827426 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 789/1596] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:12.828532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087036a4-c639-477b-9279-84acddf42f20 service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:12.838477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2bdf0-77a3-42b0-a7bb-bb4806fcd75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:12.839135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2bdf0-77a3-42b0-a7bb-bb4806fcd75a admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:12.839359 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087036a4-c639-477b-9279-84acddf42f20 service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:12.839468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2bdf0-77a3-42b0-a7bb-bb4806fcd75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:12.840474 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e2bdf0-77a3-42b0-a7bb-bb4806fcd75a admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:12.841191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e2bdf0-77a3-42b0-a7bb-bb4806fcd75a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:12.842854 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e2bdf0-77a3-42b0-a7bb-bb4806fcd75a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:12.845302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce806d-eed9-4ed8-a01f-1091cdab480b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:12.846036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce806d-eed9-4ed8-a01f-1091cdab480b admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:12.846392 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce806d-eed9-4ed8-a01f-1091cdab480b admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:12.846876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce806d-eed9-4ed8-a01f-1091cdab480b admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:12.847509 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce806d-eed9-4ed8-a01f-1091cdab480b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:12.849210 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce806d-eed9-4ed8-a01f-1091cdab480b admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:12.856034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 808/1597] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:12.857008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2bdf0-77a3-42b0-a7bb-bb4806fcd75a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:12.861481 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 790/1598] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:12.868810 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce806d-eed9-4ed8-a01f-1091cdab480b admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:12.872993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e2bdf0-77a3-42b0-a7bb-bb4806fcd75a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:12.879352 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce806d-eed9-4ed8-a01f-1091cdab480b admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:12.886120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d763f-9b7f-4134-90c3-269b06dafb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:12.887142 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f458de7-2fe9-4c94-be1d-c6fcf67e5525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:12.887360 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d763f-9b7f-4134-90c3-269b06dafb10 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:12.887633 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d763f-9b7f-4134-90c3-269b06dafb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:12.887815 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f458de7-2fe9-4c94-be1d-c6fcf67e5525 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:12.887948 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d763f-9b7f-4134-90c3-269b06dafb10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:12.888040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f458de7-2fe9-4c94-be1d-c6fcf67e5525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:12.888311 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f458de7-2fe9-4c94-be1d-c6fcf67e5525 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:12.888606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d763f-9b7f-4134-90c3-269b06dafb10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:12.889116 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f458de7-2fe9-4c94-be1d-c6fcf67e5525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:12.890416 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d763f-9b7f-4134-90c3-269b06dafb10 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:12.890762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f458de7-2fe9-4c94-be1d-c6fcf67e5525 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:12.901808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 809/1599] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:12.904643 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d763f-9b7f-4134-90c3-269b06dafb10 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:12.907446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 791/1600] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:12.908411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f458de7-2fe9-4c94-be1d-c6fcf67e5525 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:12.925699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f458de7-2fe9-4c94-be1d-c6fcf67e5525 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:12.930730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d763f-9b7f-4134-90c3-269b06dafb10 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:12.932695 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b0cc4-ad4e-480d-8950-8ffac17223f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:12.933453 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b0cc4-ad4e-480d-8950-8ffac17223f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:12.933786 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b0cc4-ad4e-480d-8950-8ffac17223f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:12.934102 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b0cc4-ad4e-480d-8950-8ffac17223f5 admin admin] PATH_INFO: `/v3/projects/4f00d6f249f94245a6e012c3740ff646/users/5e2351a44ccf4f85b2d9b9ab92657e56/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:12.941121 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaff0b6-6252-4abe-9a49-8d4cc2c7574f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:12.942571 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaff0b6-6252-4abe-9a49-8d4cc2c7574f admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:12.942889 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaff0b6-6252-4abe-9a49-8d4cc2c7574f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:12.943373 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaff0b6-6252-4abe-9a49-8d4cc2c7574f admin admin] PATH_INFO: `/v3/projects/259e864bb08645c79dfab23aa56d8822/users/7fd6b148658e4c2884a32f66326833b6/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:12.970875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaff0b6-6252-4abe-9a49-8d4cc2c7574f admin admin] RBAC: Authorizing `identity:create_grant(project_id=259e864bb08645c79dfab23aa56d8822, user_id=7fd6b148658e4c2884a32f66326833b6, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:12.973690 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaff0b6-6252-4abe-9a49-8d4cc2c7574f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:12.976416 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b0cc4-ad4e-480d-8950-8ffac17223f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f00d6f249f94245a6e012c3740ff646, user_id=5e2351a44ccf4f85b2d9b9ab92657e56, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:12.979193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3b0cc4-ad4e-480d-8950-8ffac17223f5 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:13.080040 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 810/1601] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:12 2022] PUT /identity/v3/projects/259e864bb08645c79dfab23aa56d8822/users/7fd6b148658e4c2884a32f66326833b6/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:13.083405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaff0b6-6252-4abe-9a49-8d4cc2c7574f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:13.141986 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 792/1602] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:12 2022] PUT /identity/v3/projects/4f00d6f249f94245a6e012c3740ff646/users/5e2351a44ccf4f85b2d9b9ab92657e56/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:13.142863 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b0cc4-ad4e-480d-8950-8ffac17223f5 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:13.149780 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaff0b6-6252-4abe-9a49-8d4cc2c7574f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:13.213549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:13.247991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b0cc4-ad4e-480d-8950-8ffac17223f5 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:13.254928 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0d011-350f-418b-9ee4-2556a903c3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:13.255744 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0d011-350f-418b-9ee4-2556a903c3cf admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:13.256155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0d011-350f-418b-9ee4-2556a903c3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:13.256586 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0d011-350f-418b-9ee4-2556a903c3cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:13.257553 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa0d011-350f-418b-9ee4-2556a903c3cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:13.257972 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d3a414-8c13-4ef3-bbce-84b21df7ddd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:13.258724 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3a414-8c13-4ef3-bbce-84b21df7ddd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:13.266352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3a414-8c13-4ef3-bbce-84b21df7ddd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:13.266352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d3a414-8c13-4ef3-bbce-84b21df7ddd2 admin admin] PATH_INFO: `/v3/projects/66edd4322b074654b865518b30a0a48e/users/a387e34d4254471cac5af552d2b8eb3f/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:13.266645 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa0d011-350f-418b-9ee4-2556a903c3cf admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:13.276051 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 793/1603] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:13.277029 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0d011-350f-418b-9ee4-2556a903c3cf admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:13.282679 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d3a414-8c13-4ef3-bbce-84b21df7ddd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66edd4322b074654b865518b30a0a48e, user_id=a387e34d4254471cac5af552d2b8eb3f, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:13.286567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d3a414-8c13-4ef3-bbce-84b21df7ddd2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:13.289741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0d011-350f-418b-9ee4-2556a903c3cf admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:13.295950 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae397a4-3869-41fc-b548-d2cab8ef2b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:13.303462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae397a4-3869-41fc-b548-d2cab8ef2b00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:13.303462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae397a4-3869-41fc-b548-d2cab8ef2b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:13.303462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae397a4-3869-41fc-b548-d2cab8ef2b00 admin admin] PATH_INFO: `/v3/projects/da536e0330504718a72ac13c0dd286d8/users/3c6cc094a63f478aa34d6ad790c8a4d2/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:13.316491 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae397a4-3869-41fc-b548-d2cab8ef2b00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da536e0330504718a72ac13c0dd286d8, user_id=3c6cc094a63f478aa34d6ad790c8a4d2, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:13.319503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae397a4-3869-41fc-b548-d2cab8ef2b00 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:13.403353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 811/1604] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:13 2022] PUT /identity/v3/projects/66edd4322b074654b865518b30a0a48e/users/a387e34d4254471cac5af552d2b8eb3f/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:13.403353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d3a414-8c13-4ef3-bbce-84b21df7ddd2 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:13.437927 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 794/1605] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:13 2022] PUT /identity/v3/projects/da536e0330504718a72ac13c0dd286d8/users/3c6cc094a63f478aa34d6ad790c8a4d2/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:13.438476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1c169-8440-48c5-b41f-68dbd6b3c488 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:13.446559 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1c169-8440-48c5-b41f-68dbd6b3c488 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:13.446559 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1c169-8440-48c5-b41f-68dbd6b3c488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:13.469451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-bed1c169-8440-48c5-b41f-68dbd6b3c488 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:13.471081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-bed1c169-8440-48c5-b41f-68dbd6b3c488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:13.504605 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d3a414-8c13-4ef3-bbce-84b21df7ddd2 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:13.515882 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:13.594529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 795/1606] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:13.599276 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcef36-0bb9-4c59-854a-f0f31932fa12 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:13.601042 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcef36-0bb9-4c59-854a-f0f31932fa12 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:13.601360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bcef36-0bb9-4c59-854a-f0f31932fa12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:13.634589 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef742bfe-d9f2-433e-b901-c0c13745c821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:13.635317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef742bfe-d9f2-433e-b901-c0c13745c821 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:13.635640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef742bfe-d9f2-433e-b901-c0c13745c821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:13.635922 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef742bfe-d9f2-433e-b901-c0c13745c821 admin admin] PATH_INFO: `/v3/projects/edc8f82022af4e398c2c0fbfe555ef88/users/87af3620e39a4009905152fcda5a8e23/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:13.659061 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-83bcef36-0bb9-4c59-854a-f0f31932fa12 None None] MFA Rules not processed for user `5e2351a44ccf4f85b2d9b9ab92657e56`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:13.659984 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef742bfe-d9f2-433e-b901-c0c13745c821 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edc8f82022af4e398c2c0fbfe555ef88, user_id=87af3620e39a4009905152fcda5a8e23, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:13.661207 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-83bcef36-0bb9-4c59-854a-f0f31932fa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:13.664381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef742bfe-d9f2-433e-b901-c0c13745c821 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:13.742918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 812/1607] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:13 2022] PUT /identity/v3/projects/edc8f82022af4e398c2c0fbfe555ef88/users/87af3620e39a4009905152fcda5a8e23/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:13.743815 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef742bfe-d9f2-433e-b901-c0c13745c821 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:13.758833 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 796/1608] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:13.760305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ad8ed-18e6-4169-a7ee-37b54652d55d None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:13.765373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ad8ed-18e6-4169-a7ee-37b54652d55d None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:13.765373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ad8ed-18e6-4169-a7ee-37b54652d55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:13.816568 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-537ad8ed-18e6-4169-a7ee-37b54652d55d None None] MFA Rules not processed for user `a387e34d4254471cac5af552d2b8eb3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:13.818420 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-537ad8ed-18e6-4169-a7ee-37b54652d55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:13.850482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:13.874523 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef742bfe-d9f2-433e-b901-c0c13745c821 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:13.880488 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4c381-2c1e-4581-8a8d-6f709a3a0715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:13.881224 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4c381-2c1e-4581-8a8d-6f709a3a0715 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:13.881609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4c381-2c1e-4581-8a8d-6f709a3a0715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:13.882178 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4c381-2c1e-4581-8a8d-6f709a3a0715 admin admin] PATH_INFO: `/v3/projects/b198992cc0014c6a9c25374d24fb01a9/users/3b9d9cfdcaac473786d3017f73163274/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:13.904143 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4c381-2c1e-4581-8a8d-6f709a3a0715 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b198992cc0014c6a9c25374d24fb01a9, user_id=3b9d9cfdcaac473786d3017f73163274, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:13.908174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4c381-2c1e-4581-8a8d-6f709a3a0715 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:13.944046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 797/1609] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:13.946140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055f4bc-3a8c-4372-b780-322e672575c4 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:13.946607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055f4bc-3a8c-4372-b780-322e672575c4 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:13.947273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055f4bc-3a8c-4372-b780-322e672575c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:13.983959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-1055f4bc-3a8c-4372-b780-322e672575c4 None None] MFA Rules not processed for user `3c6cc094a63f478aa34d6ad790c8a4d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:13.985186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-1055f4bc-3a8c-4372-b780-322e672575c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:13.998040 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 813/1610] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:13 2022] PUT /identity/v3/projects/b198992cc0014c6a9c25374d24fb01a9/users/3b9d9cfdcaac473786d3017f73163274/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:13.999395 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4c381-2c1e-4581-8a8d-6f709a3a0715 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:14.025103 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:14.058671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 798/1611] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:13 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:14.058671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a393424-e80e-47fb-95b0-c1d75275dd34 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:14.058671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a393424-e80e-47fb-95b0-c1d75275dd34 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:14.058671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a393424-e80e-47fb-95b0-c1d75275dd34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:14.062279 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4c381-2c1e-4581-8a8d-6f709a3a0715 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:14.069493 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dfc1aa-76f9-41f8-a4f0-eacf8509d6e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:14.070256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfc1aa-76f9-41f8-a4f0-eacf8509d6e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:14.070625 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfc1aa-76f9-41f8-a4f0-eacf8509d6e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:14.070935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfc1aa-76f9-41f8-a4f0-eacf8509d6e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:14.093299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-6a393424-e80e-47fb-95b0-c1d75275dd34 None None] MFA Rules not processed for user `87af3620e39a4009905152fcda5a8e23`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:14.094709 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-6a393424-e80e-47fb-95b0-c1d75275dd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:14.194939 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dfc1aa-76f9-41f8-a4f0-eacf8509d6e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:14.198249 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dfc1aa-76f9-41f8-a4f0-eacf8509d6e1 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:14.206023 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 799/1612] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:14.207408 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a393424-e80e-47fb-95b0-c1d75275dd34 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:14.216862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 814/1613] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:14.218671 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf45f5-aa6a-4711-a8e3-e6937a1d0a85 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:14.222602 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf45f5-aa6a-4711-a8e3-e6937a1d0a85 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:14.222895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf45f5-aa6a-4711-a8e3-e6937a1d0a85 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:14.227314 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 815/1614] 10.222.0.46 () {54 vars in 962 bytes} [Tue Aug 2 09:06:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:06:14.227314 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f8271-e126-4063-a67c-213f9951d2bc None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:14.227314 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f8271-e126-4063-a67c-213f9951d2bc None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:14.227314 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f8271-e126-4063-a67c-213f9951d2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:14.245727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a393424-e80e-47fb-95b0-c1d75275dd34 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:14.261168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-301f8271-e126-4063-a67c-213f9951d2bc None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:14.263258 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-301f8271-e126-4063-a67c-213f9951d2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:14.265753 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2be4ef-480d-4132-865d-67668eb03b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:14.266768 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2be4ef-480d-4132-865d-67668eb03b97 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:14.267128 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2be4ef-480d-4132-865d-67668eb03b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:14.270149 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2be4ef-480d-4132-865d-67668eb03b97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:14.270936 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2be4ef-480d-4132-865d-67668eb03b97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:14.272388 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 816/1615] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:14.273075 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2be4ef-480d-4132-865d-67668eb03b97 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:14.273392 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f8271-e126-4063-a67c-213f9951d2bc None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:14.285609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f8271-e126-4063-a67c-213f9951d2bc None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:14.294254 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2a352-1434-4105-a2f4-6f56c2d4108d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:14.294982 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2a352-1434-4105-a2f4-6f56c2d4108d admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:14.295266 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2a352-1434-4105-a2f4-6f56c2d4108d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:14.295540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2a352-1434-4105-a2f4-6f56c2d4108d admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:14.296219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2a352-1434-4105-a2f4-6f56c2d4108d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:14.298177 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2a352-1434-4105-a2f4-6f56c2d4108d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:14.405417 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 817/1616] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:06:14 2022] POST /identity/v3/projects => generated 403 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:14.405417 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a5239-f1fa-4a4d-8889-73b20022df46 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:14.405417 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a5239-f1fa-4a4d-8889-73b20022df46 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:14.405417 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a5239-f1fa-4a4d-8889-73b20022df46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:14.418906 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 800/1617] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:06:14 2022] POST /identity/v3/projects => generated 365 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:14.422287 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173a97e-88be-4d30-87b3-0b083ae0c2de None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:14.423009 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173a97e-88be-4d30-87b3-0b083ae0c2de None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:14.423446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0173a97e-88be-4d30-87b3-0b083ae0c2de None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:14.432991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 801/1618] 10.222.0.46 () {54 vars in 962 bytes} [Tue Aug 2 09:06:14 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:06:14.434329 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173a97e-88be-4d30-87b3-0b083ae0c2de None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:14.496919 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-d32a5239-f1fa-4a4d-8889-73b20022df46 None None] MFA Rules not processed for user `3191d804e8dc4b1a85f490319833ec07`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:14.498457 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d32a5239-f1fa-4a4d-8889-73b20022df46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:14.517616 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0173a97e-88be-4d30-87b3-0b083ae0c2de None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:14.523088 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fd8c5-28d6-4af4-bf76-2c994c020eee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:14.524711 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fd8c5-28d6-4af4-bf76-2c994c020eee service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:14.525101 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fd8c5-28d6-4af4-bf76-2c994c020eee service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:14.525824 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fd8c5-28d6-4af4-bf76-2c994c020eee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:14.549228 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:14.600903 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 818/1619] 10.222.0.46 () {58 vars in 1059 bytes} [Tue Aug 2 09:06:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:14.601832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a5239-f1fa-4a4d-8889-73b20022df46 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:14.602636 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:14.605442 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fd8c5-28d6-4af4-bf76-2c994c020eee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:14.607240 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fd8c5-28d6-4af4-bf76-2c994c020eee service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:14.613570 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a5239-f1fa-4a4d-8889-73b20022df46 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:14.619503 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6179efc8-2eec-4abd-91ea-5cad92482477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:14.619791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 802/1620] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:14.620373 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179efc8-2eec-4abd-91ea-5cad92482477 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:14.621246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179efc8-2eec-4abd-91ea-5cad92482477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:14.621605 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6179efc8-2eec-4abd-91ea-5cad92482477 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:14.622169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fd8c5-28d6-4af4-bf76-2c994c020eee service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:14.622496 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179efc8-2eec-4abd-91ea-5cad92482477 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:14.624574 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6179efc8-2eec-4abd-91ea-5cad92482477 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:14.661806 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fd8c5-28d6-4af4-bf76-2c994c020eee service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:14.669565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c065b6-4200-4926-9237-222e9fc516b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:14.672894 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c065b6-4200-4926-9237-222e9fc516b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:14.672894 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c065b6-4200-4926-9237-222e9fc516b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:14.672894 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c065b6-4200-4926-9237-222e9fc516b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:14.672894 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c065b6-4200-4926-9237-222e9fc516b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:14.675901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c065b6-4200-4926-9237-222e9fc516b3 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:14.783823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 819/1621] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:06:14 2022] POST /identity/v3/users => generated 308 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:14.788244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ab471-2017-4094-aa62-8b37d78d6f1f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:14.788544 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ab471-2017-4094-aa62-8b37d78d6f1f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:14.789051 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ab471-2017-4094-aa62-8b37d78d6f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:14.814423 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 803/1622] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:06:14 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:14.818323 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c065b6-4200-4926-9237-222e9fc516b3 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:14.820963 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-3f3ab471-2017-4094-aa62-8b37d78d6f1f None None] MFA Rules not processed for user `3191d804e8dc4b1a85f490319833ec07`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:14.823036 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3f3ab471-2017-4094-aa62-8b37d78d6f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:14.838252 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c065b6-4200-4926-9237-222e9fc516b3 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:14.847909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56e8d6-00d7-40a2-a251-96dfb56d11e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:14.848563 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56e8d6-00d7-40a2-a251-96dfb56d11e6 service nova] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:14.848853 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56e8d6-00d7-40a2-a251-96dfb56d11e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:14.849609 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c56e8d6-00d7-40a2-a251-96dfb56d11e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:14.851544 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 820/1623] 10.222.0.46 () {58 vars in 1059 bytes} [Tue Aug 2 09:06:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:14.852460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ab471-2017-4094-aa62-8b37d78d6f1f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:14.881095 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ab471-2017-4094-aa62-8b37d78d6f1f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:14.898143 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17985be-15db-4d48-aa09-1ac3159fe11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:14.901449 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17985be-15db-4d48-aa09-1ac3159fe11e admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:14.901449 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17985be-15db-4d48-aa09-1ac3159fe11e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:14.901449 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17985be-15db-4d48-aa09-1ac3159fe11e admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:14.901449 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17985be-15db-4d48-aa09-1ac3159fe11e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:14.903075 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17985be-15db-4d48-aa09-1ac3159fe11e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:14.920785 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 821/1624] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:14.923260 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17985be-15db-4d48-aa09-1ac3159fe11e admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:14.938997 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56e8d6-00d7-40a2-a251-96dfb56d11e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:14.938997 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c56e8d6-00d7-40a2-a251-96dfb56d11e6 service nova] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:14.950870 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17985be-15db-4d48-aa09-1ac3159fe11e admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:14.960619 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 804/1625] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:14.960619 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56e8d6-00d7-40a2-a251-96dfb56d11e6 service nova] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:14.967772 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb1779-3d06-44ac-939e-d1a8df2553cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:14.968656 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb1779-3d06-44ac-939e-d1a8df2553cf admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:14.968961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb1779-3d06-44ac-939e-d1a8df2553cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:14.969229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb1779-3d06-44ac-939e-d1a8df2553cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:14.970035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb1779-3d06-44ac-939e-d1a8df2553cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:14.972362 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb1779-3d06-44ac-939e-d1a8df2553cf admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:15.000247 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c56e8d6-00d7-40a2-a251-96dfb56d11e6 service nova] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:15.000817 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 822/1626] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:15.000817 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb1779-3d06-44ac-939e-d1a8df2553cf admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:15.002089 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbaa867-f5a6-4259-bc6c-7c9b1262e045 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:15.005671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbaa867-f5a6-4259-bc6c-7c9b1262e045 service nova] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:15.006017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbaa867-f5a6-4259-bc6c-7c9b1262e045 service nova] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:15.006412 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbaa867-f5a6-4259-bc6c-7c9b1262e045 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:15.019422 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb1779-3d06-44ac-939e-d1a8df2553cf admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:15.028012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadefb03-7a1c-4ad6-bfa1-f3eda41895f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:15.028804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadefb03-7a1c-4ad6-bfa1-f3eda41895f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:15.028994 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadefb03-7a1c-4ad6-bfa1-f3eda41895f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:15.029734 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadefb03-7a1c-4ad6-bfa1-f3eda41895f5 admin admin] PATH_INFO: `/v3/projects/dae89d3f4f134aca84c2b8fcdbea859d/users/7abf2a7ba0f04f1c92773844f54f3698/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:15.056291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadefb03-7a1c-4ad6-bfa1-f3eda41895f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae89d3f4f134aca84c2b8fcdbea859d, user_id=7abf2a7ba0f04f1c92773844f54f3698, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:15.062528 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadefb03-7a1c-4ad6-bfa1-f3eda41895f5 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:15.122420 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbaa867-f5a6-4259-bc6c-7c9b1262e045 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:15.124479 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbaa867-f5a6-4259-bc6c-7c9b1262e045 service nova] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:15.136713 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 805/1627] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:15.151354 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbaa867-f5a6-4259-bc6c-7c9b1262e045 service nova] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:15.181705 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbaa867-f5a6-4259-bc6c-7c9b1262e045 service nova] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:15.193009 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13568fb-9734-4d23-84c3-43e70a9bf4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:15.193837 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13568fb-9734-4d23-84c3-43e70a9bf4d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:15.194663 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13568fb-9734-4d23-84c3-43e70a9bf4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:15.194663 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13568fb-9734-4d23-84c3-43e70a9bf4d3 admin admin] PATH_INFO: `/v3/projects/5989a0169d704e24a18d61271134656c/users/cb983eb44d244da28614f9b3daaab2a6/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:15.197802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 823/1628] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:14 2022] PUT /identity/v3/projects/dae89d3f4f134aca84c2b8fcdbea859d/users/7abf2a7ba0f04f1c92773844f54f3698/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:15.201780 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadefb03-7a1c-4ad6-bfa1-f3eda41895f5 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:15.234945 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13568fb-9734-4d23-84c3-43e70a9bf4d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5989a0169d704e24a18d61271134656c, user_id=cb983eb44d244da28614f9b3daaab2a6, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:15.251976 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13568fb-9734-4d23-84c3-43e70a9bf4d3 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:15.303824 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadefb03-7a1c-4ad6-bfa1-f3eda41895f5 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:15.309510 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db40d60-c24a-4f4f-8fbc-c3a66d392830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:15.311287 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db40d60-c24a-4f4f-8fbc-c3a66d392830 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:15.311959 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db40d60-c24a-4f4f-8fbc-c3a66d392830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:15.312498 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db40d60-c24a-4f4f-8fbc-c3a66d392830 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:15.313461 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db40d60-c24a-4f4f-8fbc-c3a66d392830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:15.315425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db40d60-c24a-4f4f-8fbc-c3a66d392830 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:15.330589 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 824/1629] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:15.334887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db40d60-c24a-4f4f-8fbc-c3a66d392830 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:15.340327 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 806/1630] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:15 2022] PUT /identity/v3/projects/5989a0169d704e24a18d61271134656c/users/cb983eb44d244da28614f9b3daaab2a6/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:15.343971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13568fb-9734-4d23-84c3-43e70a9bf4d3 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:15.349751 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db40d60-c24a-4f4f-8fbc-c3a66d392830 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:15.429268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4659a-a00b-4a85-bfeb-fae1ce8a4527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:15.430217 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4659a-a00b-4a85-bfeb-fae1ce8a4527 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:15.430690 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4659a-a00b-4a85-bfeb-fae1ce8a4527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:15.431111 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4659a-a00b-4a85-bfeb-fae1ce8a4527 admin admin] PATH_INFO: `/v3/projects/dae89d3f4f134aca84c2b8fcdbea859d/users/7abf2a7ba0f04f1c92773844f54f3698/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:15.436043 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4659a-a00b-4a85-bfeb-fae1ce8a4527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae89d3f4f134aca84c2b8fcdbea859d, user_id=7abf2a7ba0f04f1c92773844f54f3698, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:15.439789 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4659a-a00b-4a85-bfeb-fae1ce8a4527 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:15.455315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 825/1631] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:15 2022] PUT /identity/v3/projects/dae89d3f4f134aca84c2b8fcdbea859d/users/7abf2a7ba0f04f1c92773844f54f3698/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:15.467765 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686c70e-3668-4212-97cf-684aa2442a89 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:15.468451 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686c70e-3668-4212-97cf-684aa2442a89 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:15.469024 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686c70e-3668-4212-97cf-684aa2442a89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:15.527957 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13568fb-9734-4d23-84c3-43e70a9bf4d3 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:15.589541 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-6686c70e-3668-4212-97cf-684aa2442a89 None None] MFA Rules not processed for user `7abf2a7ba0f04f1c92773844f54f3698`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:15.592240 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6686c70e-3668-4212-97cf-684aa2442a89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:15.661580 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d30ff-8f4b-4b8b-b4fc-13cb0acf4456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:15.662751 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d30ff-8f4b-4b8b-b4fc-13cb0acf4456 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:15.663277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d30ff-8f4b-4b8b-b4fc-13cb0acf4456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:15.663415 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d30ff-8f4b-4b8b-b4fc-13cb0acf4456 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:15.664181 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d30ff-8f4b-4b8b-b4fc-13cb0acf4456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:15.665926 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312d30ff-8f4b-4b8b-b4fc-13cb0acf4456 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:15.679057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 807/1632] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:15 2022] GET /identity/v3/roles => generated 1420 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:15.680683 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29a625-e03b-4526-b1f9-674e77786aba None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:15.681703 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29a625-e03b-4526-b1f9-674e77786aba None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:15.682014 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d29a625-e03b-4526-b1f9-674e77786aba None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:15.684394 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 808/1633] 10.222.0.46 () {54 vars in 912 bytes} [Tue Aug 2 09:06:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:06:15.685683 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4ce7d-fa46-4b3c-bd2a-710402a7dc30 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:15.686002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4ce7d-fa46-4b3c-bd2a-710402a7dc30 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:15.686259 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f4ce7d-fa46-4b3c-bd2a-710402a7dc30 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:15.687934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 809/1634] 10.222.0.46 () {54 vars in 912 bytes} [Tue Aug 2 09:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:06:15.689110 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4ce7d-fa46-4b3c-bd2a-710402a7dc30 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:15.706879 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f4ce7d-fa46-4b3c-bd2a-710402a7dc30 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:15.712686 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3090d-d9de-48e5-9780-c4b6ca4d4d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:15.713779 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3090d-d9de-48e5-9780-c4b6ca4d4d96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:15.714152 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3090d-d9de-48e5-9780-c4b6ca4d4d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:15.714489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3090d-d9de-48e5-9780-c4b6ca4d4d96 admin admin] PATH_INFO: `/v3/projects/5989a0169d704e24a18d61271134656c/users/cb983eb44d244da28614f9b3daaab2a6/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:15.716822 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c3090d-d9de-48e5-9780-c4b6ca4d4d96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5989a0169d704e24a18d61271134656c, user_id=cb983eb44d244da28614f9b3daaab2a6, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:15.719655 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c3090d-d9de-48e5-9780-c4b6ca4d4d96 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:15.737276 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 810/1635] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:15 2022] PUT /identity/v3/projects/5989a0169d704e24a18d61271134656c/users/cb983eb44d244da28614f9b3daaab2a6/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:15.737276 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ecc446-087b-4569-8f7f-b23ef02b3fc8 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:15.737276 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ecc446-087b-4569-8f7f-b23ef02b3fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:15.737276 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ecc446-087b-4569-8f7f-b23ef02b3fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:15.771574 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-a5ecc446-087b-4569-8f7f-b23ef02b3fc8 None None] MFA Rules not processed for user `3191d804e8dc4b1a85f490319833ec07`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:15.773755 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-a5ecc446-087b-4569-8f7f-b23ef02b3fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:15.800142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 826/1636] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:15 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:06:15.816290 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6a2c6-25bd-422d-a5d0-5dfef9e5f7ea None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:15.819582 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6a2c6-25bd-422d-a5d0-5dfef9e5f7ea None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:15.821211 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e6a2c6-25bd-422d-a5d0-5dfef9e5f7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:15.866528 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 811/1637] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 2 09:06:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:15.868143 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2dbfc-abe5-4d25-8294-5042d5666b97 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:15.868562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2dbfc-abe5-4d25-8294-5042d5666b97 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:15.868838 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2dbfc-abe5-4d25-8294-5042d5666b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:15.875994 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-18e6a2c6-25bd-422d-a5d0-5dfef9e5f7ea None None] MFA Rules not processed for user `3191d804e8dc4b1a85f490319833ec07`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:15.877386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-18e6a2c6-25bd-422d-a5d0-5dfef9e5f7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:15.903033 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-7ae2dbfc-abe5-4d25-8294-5042d5666b97 None None] MFA Rules not processed for user `cb983eb44d244da28614f9b3daaab2a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:15.904718 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-7ae2dbfc-abe5-4d25-8294-5042d5666b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:15.907518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:15.910285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 827/1638] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 2 09:06:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:15.911598 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6a2c6-25bd-422d-a5d0-5dfef9e5f7ea None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:15.944633 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:16.030634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e6a2c6-25bd-422d-a5d0-5dfef9e5f7ea None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:16.035243 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec0312-ea76-42f7-a59a-b60bdb3b459c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:16.035879 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec0312-ea76-42f7-a59a-b60bdb3b459c service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:16.036159 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec0312-ea76-42f7-a59a-b60bdb3b459c service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:16.036459 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ec0312-ea76-42f7-a59a-b60bdb3b459c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:16.051793 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 812/1639] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:15 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:16.053375 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2dbfc-abe5-4d25-8294-5042d5666b97 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:16.071054 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2dbfc-abe5-4d25-8294-5042d5666b97 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:16.077660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5972cac8-1a33-42a8-95a0-5d85c8e87286 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:16.077908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972cac8-1a33-42a8-95a0-5d85c8e87286 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:16.078090 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972cac8-1a33-42a8-95a0-5d85c8e87286 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:16.078348 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972cac8-1a33-42a8-95a0-5d85c8e87286 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:16.096333 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5972cac8-1a33-42a8-95a0-5d85c8e87286 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:16.097794 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5972cac8-1a33-42a8-95a0-5d85c8e87286 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:16.126844 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 813/1640] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:16.172536 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec0312-ea76-42f7-a59a-b60bdb3b459c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:16.174265 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ec0312-ea76-42f7-a59a-b60bdb3b459c service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:16.204860 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 828/1641] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:15 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:16.247522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5972cac8-1a33-42a8-95a0-5d85c8e87286 service cinder] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:16.503982 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5972cac8-1a33-42a8-95a0-5d85c8e87286 service cinder] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:16.503982 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b2115-55fc-4e53-be9a-ea1bdda8c8ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:16.503982 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b2115-55fc-4e53-be9a-ea1bdda8c8ba service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:16.503982 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b2115-55fc-4e53-be9a-ea1bdda8c8ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:16.506273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b2115-55fc-4e53-be9a-ea1bdda8c8ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:16.506273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b2115-55fc-4e53-be9a-ea1bdda8c8ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:16.506273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b2115-55fc-4e53-be9a-ea1bdda8c8ba service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:16.506273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 814/1642] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:16.510803 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec0312-ea76-42f7-a59a-b60bdb3b459c service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:16.510803 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ec0312-ea76-42f7-a59a-b60bdb3b459c service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:16.510803 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123c552-0abe-411d-95f4-af18c4b4f21a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:16.510803 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123c552-0abe-411d-95f4-af18c4b4f21a service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:16.512898 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123c552-0abe-411d-95f4-af18c4b4f21a service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:16.512898 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123c552-0abe-411d-95f4-af18c4b4f21a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:16.512898 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123c552-0abe-411d-95f4-af18c4b4f21a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:16.512898 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e123c552-0abe-411d-95f4-af18c4b4f21a service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:16.512898 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 829/1643] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:16 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:06:17.272164 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b2115-55fc-4e53-be9a-ea1bdda8c8ba service cinder] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:17.280884 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123c552-0abe-411d-95f4-af18c4b4f21a service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:17.293394 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123c552-0abe-411d-95f4-af18c4b4f21a service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:17.308335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff4dcd-73c4-4e7a-bd70-741e20020523 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:17.308971 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff4dcd-73c4-4e7a-bd70-741e20020523 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:17.309255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff4dcd-73c4-4e7a-bd70-741e20020523 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:17.309531 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff4dcd-73c4-4e7a-bd70-741e20020523 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:17.365569 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:17.433684 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b2115-55fc-4e53-be9a-ea1bdda8c8ba service cinder] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:17.436716 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff4dcd-73c4-4e7a-bd70-741e20020523 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:17.439518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff4dcd-73c4-4e7a-bd70-741e20020523 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:17.445505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:17.450799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47de233-8a2c-4108-8d1b-3f725655f00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:17.451841 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47de233-8a2c-4108-8d1b-3f725655f00e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:17.452321 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47de233-8a2c-4108-8d1b-3f725655f00e admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:17.452909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47de233-8a2c-4108-8d1b-3f725655f00e admin admin] PATH_INFO: `/v3/users/a387e34d4254471cac5af552d2b8eb3f` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:17.455072 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47de233-8a2c-4108-8d1b-3f725655f00e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a387e34d4254471cac5af552d2b8eb3f)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:17.459592 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 830/1644] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:17.468579 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47de233-8a2c-4108-8d1b-3f725655f00e admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:17.793515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff4dcd-73c4-4e7a-bd70-741e20020523 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:17.816393 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff4dcd-73c4-4e7a-bd70-741e20020523 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:17.822449 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-d47de233-8a2c-4108-8d1b-3f725655f00e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a387e34d4254471cac5af552d2b8eb3f'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:17.830067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9df1bb-33ae-4628-b41f-2e1c36aa566c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:17.835722 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9df1bb-33ae-4628-b41f-2e1c36aa566c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:17.835722 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9df1bb-33ae-4628-b41f-2e1c36aa566c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:17.835722 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9df1bb-33ae-4628-b41f-2e1c36aa566c admin admin] PATH_INFO: `/v3/users/3c6cc094a63f478aa34d6ad790c8a4d2` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:17.841281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9df1bb-33ae-4628-b41f-2e1c36aa566c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c6cc094a63f478aa34d6ad790c8a4d2)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:17.853002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: WARNING py.warnings [None req-ae9df1bb-33ae-4628-b41f-2e1c36aa566c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:06:17.853002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: warnings.warn(msg) Aug 02 09:06:17.853002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: Aug 02 09:06:17.855844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9df1bb-33ae-4628-b41f-2e1c36aa566c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:17.859530 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-d47de233-8a2c-4108-8d1b-3f725655f00e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a387e34d4254471cac5af552d2b8eb3f'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:17.861504 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-d47de233-8a2c-4108-8d1b-3f725655f00e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a387e34d4254471cac5af552d2b8eb3f'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:17.993848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-d47de233-8a2c-4108-8d1b-3f725655f00e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a387e34d4254471cac5af552d2b8eb3f'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:18.038209 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-d47de233-8a2c-4108-8d1b-3f725655f00e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a387e34d4254471cac5af552d2b8eb3f'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:18.046887 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 815/1645] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:06:17 2022] DELETE /identity/v3/users/a387e34d4254471cac5af552d2b8eb3f => generated 0 bytes in 775 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:18.054497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d481ac6-d70d-4086-83ac-4c9512446f7f None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:18.056203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d481ac6-d70d-4086-83ac-4c9512446f7f None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:18.056568 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d481ac6-d70d-4086-83ac-4c9512446f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:18.104256 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-7d481ac6-d70d-4086-83ac-4c9512446f7f None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:18.106638 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-7d481ac6-d70d-4086-83ac-4c9512446f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:18.189225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ae9df1bb-33ae-4628-b41f-2e1c36aa566c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c6cc094a63f478aa34d6ad790c8a4d2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:18.214363 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ae9df1bb-33ae-4628-b41f-2e1c36aa566c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c6cc094a63f478aa34d6ad790c8a4d2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:18.216604 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ae9df1bb-33ae-4628-b41f-2e1c36aa566c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c6cc094a63f478aa34d6ad790c8a4d2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:18.248871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 816/1646] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:18.268791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d481ac6-d70d-4086-83ac-4c9512446f7f None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:18.275141 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ae9df1bb-33ae-4628-b41f-2e1c36aa566c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c6cc094a63f478aa34d6ad790c8a4d2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:18.278304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ae9df1bb-33ae-4628-b41f-2e1c36aa566c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c6cc094a63f478aa34d6ad790c8a4d2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:18.279334 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-7d481ac6-d70d-4086-83ac-4c9512446f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:18.392062 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 831/1647] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:06:17 2022] DELETE /identity/v3/users/3c6cc094a63f478aa34d6ad790c8a4d2 => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:18.396619 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f4de6-25f9-4359-aa13-80f5f7b37276 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:18.397169 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f4de6-25f9-4359-aa13-80f5f7b37276 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:18.397641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f4de6-25f9-4359-aa13-80f5f7b37276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:18.444252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-b95f4de6-25f9-4359-aa13-80f5f7b37276 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:18.446449 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b95f4de6-25f9-4359-aa13-80f5f7b37276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:18.463367 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d481ac6-d70d-4086-83ac-4c9512446f7f None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:18.533957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 832/1648] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:06:18.577676 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7611a5-d019-4b0e-b0b0-1faa56ba25a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:18.578355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7611a5-d019-4b0e-b0b0-1faa56ba25a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:18.579422 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7611a5-d019-4b0e-b0b0-1faa56ba25a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:18.579716 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7611a5-d019-4b0e-b0b0-1faa56ba25a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:18.586563 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4b7611a5-d019-4b0e-b0b0-1faa56ba25a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:18.619838 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7611a5-d019-4b0e-b0b0-1faa56ba25a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:18.621865 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7611a5-d019-4b0e-b0b0-1faa56ba25a8 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:18.628975 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f4de6-25f9-4359-aa13-80f5f7b37276 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:18.630867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b95f4de6-25f9-4359-aa13-80f5f7b37276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:18.650439 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 817/1649] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:18.651035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f4de6-25f9-4359-aa13-80f5f7b37276 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:18.666773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29443c-2d6b-4c6d-911b-c4017f0b9502 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:18.668194 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29443c-2d6b-4c6d-911b-c4017f0b9502 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:18.668606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29443c-2d6b-4c6d-911b-c4017f0b9502 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:18.669005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29443c-2d6b-4c6d-911b-c4017f0b9502 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:18.691465 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29443c-2d6b-4c6d-911b-c4017f0b9502 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:18.691465 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29443c-2d6b-4c6d-911b-c4017f0b9502 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:18.717420 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 833/1650] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:18.783693 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:18.843945 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:20.497484 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7611a5-d019-4b0e-b0b0-1faa56ba25a8 service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:20.504809 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4b7611a5-d019-4b0e-b0b0-1faa56ba25a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:20.544079 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7611a5-d019-4b0e-b0b0-1faa56ba25a8 service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:20.560312 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb7008-1808-4893-9b18-fe25e46647c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:20.561075 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb7008-1808-4893-9b18-fe25e46647c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:20.561369 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb7008-1808-4893-9b18-fe25e46647c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:20.571644 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb7008-1808-4893-9b18-fe25e46647c8 admin admin] PATH_INFO: `/v3/projects/66edd4322b074654b865518b30a0a48e` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:20.571644 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb7008-1808-4893-9b18-fe25e46647c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66edd4322b074654b865518b30a0a48e)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:20.571644 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: WARNING py.warnings [None req-43cb7008-1808-4893-9b18-fe25e46647c8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 09:06:20.571644 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: warnings.warn(msg) Aug 02 09:06:20.571644 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: Aug 02 09:06:20.571644 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb7008-1808-4893-9b18-fe25e46647c8 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:20.765199 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-43cb7008-1808-4893-9b18-fe25e46647c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66edd4322b074654b865518b30a0a48e', 'request_id': 'req-43cb7008-1808-4893-9b18-fe25e46647c8'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:20.785065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-43cb7008-1808-4893-9b18-fe25e46647c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66edd4322b074654b865518b30a0a48e', 'request_id': 'req-43cb7008-1808-4893-9b18-fe25e46647c8'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:20.810017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-43cb7008-1808-4893-9b18-fe25e46647c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66edd4322b074654b865518b30a0a48e', 'request_id': 'req-43cb7008-1808-4893-9b18-fe25e46647c8'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:20.927780 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-43cb7008-1808-4893-9b18-fe25e46647c8 admin admin] The token cache is being invalidate because project 66edd4322b074654b865518b30a0a48e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:06:20.928115 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-43cb7008-1808-4893-9b18-fe25e46647c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:20.928773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-43cb7008-1808-4893-9b18-fe25e46647c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:20.930665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 818/1651] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:06:20 2022] DELETE /identity/v3/projects/66edd4322b074654b865518b30a0a48e => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:20.935653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f887e84a-6f0c-4390-83a6-604cfa02ad7f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:20.944272 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f887e84a-6f0c-4390-83a6-604cfa02ad7f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:20.944775 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f887e84a-6f0c-4390-83a6-604cfa02ad7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:20.986888 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f887e84a-6f0c-4390-83a6-604cfa02ad7f None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:20.990960 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f887e84a-6f0c-4390-83a6-604cfa02ad7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:21.162680 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 834/1652] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:21.171440 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79d26f-27ae-490d-8fe7-d9b636851160 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:21.172040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79d26f-27ae-490d-8fe7-d9b636851160 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:21.172487 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca79d26f-27ae-490d-8fe7-d9b636851160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:21.235367 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-ca79d26f-27ae-490d-8fe7-d9b636851160 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:21.239160 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-ca79d26f-27ae-490d-8fe7-d9b636851160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:21.268669 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 819/1653] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:21.273288 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f887e84a-6f0c-4390-83a6-604cfa02ad7f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:21.289783 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f887e84a-6f0c-4390-83a6-604cfa02ad7f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:21.312012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcf068-319f-474d-ac52-7f0ac3c0cb8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:21.312890 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcf068-319f-474d-ac52-7f0ac3c0cb8b admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:21.313295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcf068-319f-474d-ac52-7f0ac3c0cb8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:21.313684 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcf068-319f-474d-ac52-7f0ac3c0cb8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:21.314594 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bcf068-319f-474d-ac52-7f0ac3c0cb8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:06:21.315023 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bcf068-319f-474d-ac52-7f0ac3c0cb8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:21.316823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bcf068-319f-474d-ac52-7f0ac3c0cb8b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:21.345030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 835/1654] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:06:21.350269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79d26f-27ae-490d-8fe7-d9b636851160 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:21.364954 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca79d26f-27ae-490d-8fe7-d9b636851160 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:21.377839 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd0da5-6abf-4d3b-8d5e-3d54a78d1a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:21.378911 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd0da5-6abf-4d3b-8d5e-3d54a78d1a8a admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:21.379901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd0da5-6abf-4d3b-8d5e-3d54a78d1a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:21.379901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd0da5-6abf-4d3b-8d5e-3d54a78d1a8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:21.381492 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd0da5-6abf-4d3b-8d5e-3d54a78d1a8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:21.387101 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd0da5-6abf-4d3b-8d5e-3d54a78d1a8a admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:21.527911 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 820/1655] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:06:21 2022] POST /identity/v3/projects => generated 383 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:21.534220 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcf068-319f-474d-ac52-7f0ac3c0cb8b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:21.545690 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcf068-319f-474d-ac52-7f0ac3c0cb8b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:21.728091 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca541c1-54da-4dde-b56f-c5bd19ee50ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:21.729053 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca541c1-54da-4dde-b56f-c5bd19ee50ca admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:21.729291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca541c1-54da-4dde-b56f-c5bd19ee50ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:21.729764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca541c1-54da-4dde-b56f-c5bd19ee50ca admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:21.731093 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca541c1-54da-4dde-b56f-c5bd19ee50ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:21.741309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca541c1-54da-4dde-b56f-c5bd19ee50ca admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:21.900947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 836/1656] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:06:21 2022] POST /identity/v3/users => generated 292 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:21.905766 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd0da5-6abf-4d3b-8d5e-3d54a78d1a8a admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:21.915849 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:21.946777 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd0da5-6abf-4d3b-8d5e-3d54a78d1a8a admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:21.957276 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86373a35-f39c-4b9d-9d8f-5b019e4f13e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:21.958466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86373a35-f39c-4b9d-9d8f-5b019e4f13e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:21.961494 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86373a35-f39c-4b9d-9d8f-5b019e4f13e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:21.962525 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86373a35-f39c-4b9d-9d8f-5b019e4f13e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:21.963501 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86373a35-f39c-4b9d-9d8f-5b019e4f13e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:21.970454 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86373a35-f39c-4b9d-9d8f-5b019e4f13e9 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:21.992735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 821/1657] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:21.996748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca541c1-54da-4dde-b56f-c5bd19ee50ca admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:22.010506 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca541c1-54da-4dde-b56f-c5bd19ee50ca admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:22.024145 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b3229-7e48-4b5d-97b5-1b85f4ee56c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:22.025242 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b3229-7e48-4b5d-97b5-1b85f4ee56c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:22.025767 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b3229-7e48-4b5d-97b5-1b85f4ee56c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:22.026323 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670b3229-7e48-4b5d-97b5-1b85f4ee56c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:22.027807 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b3229-7e48-4b5d-97b5-1b85f4ee56c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:22.029685 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670b3229-7e48-4b5d-97b5-1b85f4ee56c6 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:22.062882 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 837/1658] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:22.063741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86373a35-f39c-4b9d-9d8f-5b019e4f13e9 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:22.080745 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86373a35-f39c-4b9d-9d8f-5b019e4f13e9 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:22.096492 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda267b3-8d70-4e6d-b8bf-38c02c040bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:22.100383 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda267b3-8d70-4e6d-b8bf-38c02c040bfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:22.100911 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda267b3-8d70-4e6d-b8bf-38c02c040bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:22.101343 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda267b3-8d70-4e6d-b8bf-38c02c040bfc admin admin] PATH_INFO: `/v3/projects/bef90b0b9ccb444b92430cc079fc846a/users/95347780190343d391fdc7752d171c8d/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:22.149977 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda267b3-8d70-4e6d-b8bf-38c02c040bfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bef90b0b9ccb444b92430cc079fc846a, user_id=95347780190343d391fdc7752d171c8d, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:22.153699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda267b3-8d70-4e6d-b8bf-38c02c040bfc admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:22.227378 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b3229-7e48-4b5d-97b5-1b85f4ee56c6 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:22.229560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-670b3229-7e48-4b5d-97b5-1b85f4ee56c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:22.263367 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670b3229-7e48-4b5d-97b5-1b85f4ee56c6 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:22.284025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d80658-1a46-4135-aba5-575eeca29bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:22.285031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d80658-1a46-4135-aba5-575eeca29bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:22.285829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d80658-1a46-4135-aba5-575eeca29bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:22.286628 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d80658-1a46-4135-aba5-575eeca29bd2 admin admin] PATH_INFO: `/v3/projects/da536e0330504718a72ac13c0dd286d8` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:22.294260 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d80658-1a46-4135-aba5-575eeca29bd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da536e0330504718a72ac13c0dd286d8)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:22.298929 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d80658-1a46-4135-aba5-575eeca29bd2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:22.307405 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 822/1659] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:22 2022] PUT /identity/v3/projects/bef90b0b9ccb444b92430cc079fc846a/users/95347780190343d391fdc7752d171c8d/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:22.325596 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698bda0-9b56-4ede-88ec-619eed6506be None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:22.325596 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698bda0-9b56-4ede-88ec-619eed6506be None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:22.325596 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698bda0-9b56-4ede-88ec-619eed6506be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:22.479767 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-6698bda0-9b56-4ede-88ec-619eed6506be None None] MFA Rules not processed for user `95347780190343d391fdc7752d171c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:22.481568 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-6698bda0-9b56-4ede-88ec-619eed6506be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:22.610427 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-34d80658-1a46-4135-aba5-575eeca29bd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da536e0330504718a72ac13c0dd286d8', 'request_id': 'req-34d80658-1a46-4135-aba5-575eeca29bd2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:22.634058 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-34d80658-1a46-4135-aba5-575eeca29bd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da536e0330504718a72ac13c0dd286d8', 'request_id': 'req-34d80658-1a46-4135-aba5-575eeca29bd2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:22.728925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 823/1660] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:22.733217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698bda0-9b56-4ede-88ec-619eed6506be None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:22.736626 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-34d80658-1a46-4135-aba5-575eeca29bd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da536e0330504718a72ac13c0dd286d8', 'request_id': 'req-34d80658-1a46-4135-aba5-575eeca29bd2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:22.739414 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:22.761951 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-34d80658-1a46-4135-aba5-575eeca29bd2 admin admin] The token cache is being invalidate because project da536e0330504718a72ac13c0dd286d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:06:22.769053 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-34d80658-1a46-4135-aba5-575eeca29bd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:22.773235 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-34d80658-1a46-4135-aba5-575eeca29bd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:22.786119 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 838/1661] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:06:22 2022] DELETE /identity/v3/projects/da536e0330504718a72ac13c0dd286d8 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:06:22.790286 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d80658-1a46-4135-aba5-575eeca29bd2 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:22.798197 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-34d80658-1a46-4135-aba5-575eeca29bd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:22.982401 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698bda0-9b56-4ede-88ec-619eed6506be None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:22.987642 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d80658-1a46-4135-aba5-575eeca29bd2 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:23.002459 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a526078-9d6b-4467-8988-db081adf68d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:23.003367 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a526078-9d6b-4467-8988-db081adf68d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:23.003778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a526078-9d6b-4467-8988-db081adf68d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:23.004125 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a526078-9d6b-4467-8988-db081adf68d9 admin admin] PATH_INFO: `/v3/users/7abf2a7ba0f04f1c92773844f54f3698` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:23.008223 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a526078-9d6b-4467-8988-db081adf68d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7abf2a7ba0f04f1c92773844f54f3698)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:23.009411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad487762-5f8b-433d-9275-257bbfea339a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:23.010374 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a526078-9d6b-4467-8988-db081adf68d9 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:23.010918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad487762-5f8b-433d-9275-257bbfea339a admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:23.011379 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad487762-5f8b-433d-9275-257bbfea339a admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:23.013348 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad487762-5f8b-433d-9275-257bbfea339a admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:23.014243 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad487762-5f8b-433d-9275-257bbfea339a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:23.019779 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad487762-5f8b-433d-9275-257bbfea339a admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:23.156477 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 824/1662] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:06:22 2022] POST /identity/v3/projects => generated 383 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:23.158738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad487762-5f8b-433d-9275-257bbfea339a admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:23.165408 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-ad487762-5f8b-433d-9275-257bbfea339a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:23.255518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6a526078-9d6b-4467-8988-db081adf68d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7abf2a7ba0f04f1c92773844f54f3698'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:23.284144 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6a526078-9d6b-4467-8988-db081adf68d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7abf2a7ba0f04f1c92773844f54f3698'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:23.288484 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6a526078-9d6b-4467-8988-db081adf68d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7abf2a7ba0f04f1c92773844f54f3698'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:23.335550 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6a526078-9d6b-4467-8988-db081adf68d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7abf2a7ba0f04f1c92773844f54f3698'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:23.338302 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6a526078-9d6b-4467-8988-db081adf68d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7abf2a7ba0f04f1c92773844f54f3698'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:23.364540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad487762-5f8b-433d-9275-257bbfea339a admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:23.371493 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eadd31-c8b6-4a26-a209-8a20ab11083a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:23.372449 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eadd31-c8b6-4a26-a209-8a20ab11083a admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:23.372874 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eadd31-c8b6-4a26-a209-8a20ab11083a admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:23.373278 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eadd31-c8b6-4a26-a209-8a20ab11083a admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:23.374175 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eadd31-c8b6-4a26-a209-8a20ab11083a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:23.376366 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eadd31-c8b6-4a26-a209-8a20ab11083a admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:23.484302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 825/1663] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:06:23 2022] POST /identity/v3/users => generated 292 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:23.489392 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 839/1664] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:06:22 2022] DELETE /identity/v3/users/7abf2a7ba0f04f1c92773844f54f3698 => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:23.492135 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eadd31-c8b6-4a26-a209-8a20ab11083a admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:23.497625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-d2eadd31-c8b6-4a26-a209-8a20ab11083a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:23.527693 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eadd31-c8b6-4a26-a209-8a20ab11083a admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:23.616516 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:23.719451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e7d26-764b-4af4-bddd-3fd0d41b6133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:23.719781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e7d26-764b-4af4-bddd-3fd0d41b6133 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:23.723381 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e7d26-764b-4af4-bddd-3fd0d41b6133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:23.723676 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723e7d26-764b-4af4-bddd-3fd0d41b6133 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:23.726029 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723e7d26-764b-4af4-bddd-3fd0d41b6133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:23.728806 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723e7d26-764b-4af4-bddd-3fd0d41b6133 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:23.767517 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 826/1665] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:23.768851 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a526078-9d6b-4467-8988-db081adf68d9 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:23.788141 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a526078-9d6b-4467-8988-db081adf68d9 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:23.813179 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d3fa85-dc71-4938-954d-cbd0f9193af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:23.814184 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3fa85-dc71-4938-954d-cbd0f9193af8 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:23.814537 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3fa85-dc71-4938-954d-cbd0f9193af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:23.814983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d3fa85-dc71-4938-954d-cbd0f9193af8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:23.815736 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3fa85-dc71-4938-954d-cbd0f9193af8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:23.817380 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d3fa85-dc71-4938-954d-cbd0f9193af8 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:23.857478 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 840/1666] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:23 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:23.858411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e7d26-764b-4af4-bddd-3fd0d41b6133 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:23.880709 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723e7d26-764b-4af4-bddd-3fd0d41b6133 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:23.904680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca55ef-6a5d-4ec5-8190-0587c22240b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:23.904680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca55ef-6a5d-4ec5-8190-0587c22240b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:23.904680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca55ef-6a5d-4ec5-8190-0587c22240b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:23.904680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ca55ef-6a5d-4ec5-8190-0587c22240b5 admin admin] PATH_INFO: `/v3/projects/4d3ac70cda5846aa8cd5b292dae06149/users/12dcab0775e048a09ce82a37afb2daea/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:23.974451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca55ef-6a5d-4ec5-8190-0587c22240b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d3ac70cda5846aa8cd5b292dae06149, user_id=12dcab0775e048a09ce82a37afb2daea, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:23.977760 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ca55ef-6a5d-4ec5-8190-0587c22240b5 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:24.091303 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 827/1667] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:23 2022] PUT /identity/v3/projects/4d3ac70cda5846aa8cd5b292dae06149/users/12dcab0775e048a09ce82a37afb2daea/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:06:24.102551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00870964-dc69-4bae-af7c-6f09fa065e05 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:24.102995 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00870964-dc69-4bae-af7c-6f09fa065e05 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:24.103107 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00870964-dc69-4bae-af7c-6f09fa065e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:24.171985 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-00870964-dc69-4bae-af7c-6f09fa065e05 None None] MFA Rules not processed for user `12dcab0775e048a09ce82a37afb2daea`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:24.174801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-00870964-dc69-4bae-af7c-6f09fa065e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:24.295611 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 841/1668] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:24.296244 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca55ef-6a5d-4ec5-8190-0587c22240b5 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:24.451701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ca55ef-6a5d-4ec5-8190-0587c22240b5 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:24.460563 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668c4ba-314b-440d-b126-d6ce0ad42446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:24.461404 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668c4ba-314b-440d-b126-d6ce0ad42446 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:24.461697 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668c4ba-314b-440d-b126-d6ce0ad42446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:24.462010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668c4ba-314b-440d-b126-d6ce0ad42446 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:24.464014 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668c4ba-314b-440d-b126-d6ce0ad42446 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:24.466076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4668c4ba-314b-440d-b126-d6ce0ad42446 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:24.508281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:24.590853 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 828/1669] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:06:24 2022] POST /identity/v3/projects => generated 385 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:06:24.594781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00870964-dc69-4bae-af7c-6f09fa065e05 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:24.609429 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00870964-dc69-4bae-af7c-6f09fa065e05 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:24.785212 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e4dda-082e-47f6-be03-b6c46e476b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:24.786232 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e4dda-082e-47f6-be03-b6c46e476b3b admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:24.786663 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e4dda-082e-47f6-be03-b6c46e476b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:24.787085 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e4dda-082e-47f6-be03-b6c46e476b3b admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:24.800894 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e4dda-082e-47f6-be03-b6c46e476b3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:24.805855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e4dda-082e-47f6-be03-b6c46e476b3b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:24.920575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 842/1670] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:06:24 2022] POST /identity/v3/users => generated 299 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:24.927385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668c4ba-314b-440d-b126-d6ce0ad42446 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:24.956958 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668c4ba-314b-440d-b126-d6ce0ad42446 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:24.970062 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16081ec9-70d8-4e5e-81d5-7c2ec556bd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:24.973375 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16081ec9-70d8-4e5e-81d5-7c2ec556bd3d admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:24.975741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16081ec9-70d8-4e5e-81d5-7c2ec556bd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:24.977072 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16081ec9-70d8-4e5e-81d5-7c2ec556bd3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:24.978181 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16081ec9-70d8-4e5e-81d5-7c2ec556bd3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:24.988007 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16081ec9-70d8-4e5e-81d5-7c2ec556bd3d admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:25.017963 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 829/1671] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:24 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:25.022629 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e4dda-082e-47f6-be03-b6c46e476b3b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:25.036258 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e4dda-082e-47f6-be03-b6c46e476b3b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:25.040570 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db81d2-8bd1-46c8-92b6-106d434bad5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:25.041677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db81d2-8bd1-46c8-92b6-106d434bad5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:25.042200 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db81d2-8bd1-46c8-92b6-106d434bad5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:25.042810 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72db81d2-8bd1-46c8-92b6-106d434bad5c admin admin] PATH_INFO: `/v3/projects/74a8979a91b14849a76e92d528669888/users/60aa73451a2a4c2095cbe556ca036839/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:25.061248 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72db81d2-8bd1-46c8-92b6-106d434bad5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=74a8979a91b14849a76e92d528669888, user_id=60aa73451a2a4c2095cbe556ca036839, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:25.064439 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72db81d2-8bd1-46c8-92b6-106d434bad5c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:25.153100 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16081ec9-70d8-4e5e-81d5-7c2ec556bd3d admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:25.157687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-16081ec9-70d8-4e5e-81d5-7c2ec556bd3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:25.201346 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 843/1672] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:25 2022] PUT /identity/v3/projects/74a8979a91b14849a76e92d528669888/users/60aa73451a2a4c2095cbe556ca036839/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:25.202422 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16081ec9-70d8-4e5e-81d5-7c2ec556bd3d admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:25.202837 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db81d2-8bd1-46c8-92b6-106d434bad5c admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:25.271131 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:25.452568 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae64e1-5be9-4f2c-9ef8-b8cbbcac91e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:25.456343 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae64e1-5be9-4f2c-9ef8-b8cbbcac91e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:25.458521 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae64e1-5be9-4f2c-9ef8-b8cbbcac91e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:25.458842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae64e1-5be9-4f2c-9ef8-b8cbbcac91e5 admin admin] PATH_INFO: `/v3/projects/dae89d3f4f134aca84c2b8fcdbea859d` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:25.466228 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ae64e1-5be9-4f2c-9ef8-b8cbbcac91e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dae89d3f4f134aca84c2b8fcdbea859d)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:25.469161 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ae64e1-5be9-4f2c-9ef8-b8cbbcac91e5 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:25.477393 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72db81d2-8bd1-46c8-92b6-106d434bad5c admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:25.504972 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8546fc-bca7-428c-a48c-249ef081dc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:25.506028 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8546fc-bca7-428c-a48c-249ef081dc51 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:25.507552 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8546fc-bca7-428c-a48c-249ef081dc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:25.508040 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8546fc-bca7-428c-a48c-249ef081dc51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:25.511828 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8546fc-bca7-428c-a48c-249ef081dc51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:25.515790 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8546fc-bca7-428c-a48c-249ef081dc51 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:25.592558 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 844/1673] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:25.597072 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8546fc-bca7-428c-a48c-249ef081dc51 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:25.669281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8546fc-bca7-428c-a48c-249ef081dc51 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:25.730070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-33ae64e1-5be9-4f2c-9ef8-b8cbbcac91e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dae89d3f4f134aca84c2b8fcdbea859d', 'request_id': 'req-33ae64e1-5be9-4f2c-9ef8-b8cbbcac91e5'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:25.754244 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-33ae64e1-5be9-4f2c-9ef8-b8cbbcac91e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dae89d3f4f134aca84c2b8fcdbea859d', 'request_id': 'req-33ae64e1-5be9-4f2c-9ef8-b8cbbcac91e5'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:25.785759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-33ae64e1-5be9-4f2c-9ef8-b8cbbcac91e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dae89d3f4f134aca84c2b8fcdbea859d', 'request_id': 'req-33ae64e1-5be9-4f2c-9ef8-b8cbbcac91e5'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:25.922808 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-33ae64e1-5be9-4f2c-9ef8-b8cbbcac91e5 admin admin] The token cache is being invalidate because project dae89d3f4f134aca84c2b8fcdbea859d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:06:25.923268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-33ae64e1-5be9-4f2c-9ef8-b8cbbcac91e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:25.940335 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-33ae64e1-5be9-4f2c-9ef8-b8cbbcac91e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:25.945819 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 830/1674] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:06:25 2022] DELETE /identity/v3/projects/dae89d3f4f134aca84c2b8fcdbea859d => generated 0 bytes in 793 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:25.953304 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee37d50-bc5b-4e7a-8780-98accfd52338 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:25.958817 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee37d50-bc5b-4e7a-8780-98accfd52338 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:25.959386 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee37d50-bc5b-4e7a-8780-98accfd52338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:25.987225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecd485f-af70-42d9-930a-8b5da9ecbbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:25.988244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd485f-af70-42d9-930a-8b5da9ecbbdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:25.988707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd485f-af70-42d9-930a-8b5da9ecbbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:25.989388 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecd485f-af70-42d9-930a-8b5da9ecbbdd admin admin] PATH_INFO: `/v3/projects/74a8979a91b14849a76e92d528669888/users/60aa73451a2a4c2095cbe556ca036839/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:26.007177 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd485f-af70-42d9-930a-8b5da9ecbbdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=74a8979a91b14849a76e92d528669888, user_id=60aa73451a2a4c2095cbe556ca036839, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:26.007177 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecd485f-af70-42d9-930a-8b5da9ecbbdd admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:26.041806 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-3ee37d50-bc5b-4e7a-8780-98accfd52338 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:26.047842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-3ee37d50-bc5b-4e7a-8780-98accfd52338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:26.058685 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 845/1675] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:25 2022] PUT /identity/v3/projects/74a8979a91b14849a76e92d528669888/users/60aa73451a2a4c2095cbe556ca036839/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:26.080306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af00887-502b-45e1-a456-0591657b44d2 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:26.080873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af00887-502b-45e1-a456-0591657b44d2 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:26.081356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af00887-502b-45e1-a456-0591657b44d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:26.141937 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9af00887-502b-45e1-a456-0591657b44d2 None None] MFA Rules not processed for user `60aa73451a2a4c2095cbe556ca036839`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:26.144308 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9af00887-502b-45e1-a456-0591657b44d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:26.275780 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 831/1676] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:26.283563 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9848d04-5d1e-4f08-b24d-565e5e53477c None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:26.284011 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9848d04-5d1e-4f08-b24d-565e5e53477c None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:26.284297 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9848d04-5d1e-4f08-b24d-565e5e53477c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:26.332561 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-c9848d04-5d1e-4f08-b24d-565e5e53477c None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:26.334128 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-c9848d04-5d1e-4f08-b24d-565e5e53477c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:26.364620 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 832/1677] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:26.365630 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9848d04-5d1e-4f08-b24d-565e5e53477c None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:26.381552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9848d04-5d1e-4f08-b24d-565e5e53477c None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:26.390774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf6756-7412-49d7-93f6-5182c6c8ba8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:26.394523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf6756-7412-49d7-93f6-5182c6c8ba8e admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:26.394523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf6756-7412-49d7-93f6-5182c6c8ba8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:26.394523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf6756-7412-49d7-93f6-5182c6c8ba8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:26.394523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf6756-7412-49d7-93f6-5182c6c8ba8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:06:26.394855 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf6756-7412-49d7-93f6-5182c6c8ba8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:26.396668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf6756-7412-49d7-93f6-5182c6c8ba8e admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:26.416565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 846/1678] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:26 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:06:26.422952 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 833/1679] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:06:26.424016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af00887-502b-45e1-a456-0591657b44d2 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:26.430884 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf6756-7412-49d7-93f6-5182c6c8ba8e admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:26.438231 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f3cf6756-7412-49d7-93f6-5182c6c8ba8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:26.441149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af00887-502b-45e1-a456-0591657b44d2 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:26.463740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47f4bb-68d9-418f-a604-bc96b8c666db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:26.465912 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47f4bb-68d9-418f-a604-bc96b8c666db admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:26.466521 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47f4bb-68d9-418f-a604-bc96b8c666db admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:26.467262 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc47f4bb-68d9-418f-a604-bc96b8c666db admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:26.468162 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47f4bb-68d9-418f-a604-bc96b8c666db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:26.470591 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc47f4bb-68d9-418f-a604-bc96b8c666db admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:26.533760 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:26.585684 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf6756-7412-49d7-93f6-5182c6c8ba8e admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:26.596245 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 847/1680] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:06:26 2022] POST /identity/v3/projects => generated 373 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:26.600929 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f721f7-b231-44ec-928c-6b6fab5e46d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:26.601128 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47f4bb-68d9-418f-a604-bc96b8c666db admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:26.606325 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f721f7-b231-44ec-928c-6b6fab5e46d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:26.606325 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f721f7-b231-44ec-928c-6b6fab5e46d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:26.606325 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f721f7-b231-44ec-928c-6b6fab5e46d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:26.606557 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:26.624073 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f721f7-b231-44ec-928c-6b6fab5e46d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:26.626085 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f721f7-b231-44ec-928c-6b6fab5e46d2 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:26.652528 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 834/1681] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:26 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:06:26.748181 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc47f4bb-68d9-418f-a604-bc96b8c666db admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:26.759502 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad8093-535c-449c-b846-e54cd46b6eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:26.760160 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad8093-535c-449c-b846-e54cd46b6eea admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:26.765985 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad8093-535c-449c-b846-e54cd46b6eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:26.766362 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dad8093-535c-449c-b846-e54cd46b6eea admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:26.767104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dad8093-535c-449c-b846-e54cd46b6eea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:26.773561 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dad8093-535c-449c-b846-e54cd46b6eea admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:26.961991 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 848/1682] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:06:26 2022] POST /identity/v3/users => generated 287 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:26.962245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f721f7-b231-44ec-928c-6b6fab5e46d2 service cinder] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:26.986405 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f721f7-b231-44ec-928c-6b6fab5e46d2 service cinder] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:27.009330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2472c-6d27-4f6a-baf1-df7e171f49f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:27.010055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2472c-6d27-4f6a-baf1-df7e171f49f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:27.010351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2472c-6d27-4f6a-baf1-df7e171f49f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:27.013296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2472c-6d27-4f6a-baf1-df7e171f49f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:27.013296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2472c-6d27-4f6a-baf1-df7e171f49f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:27.019460 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2472c-6d27-4f6a-baf1-df7e171f49f4 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:27.066015 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad8093-535c-449c-b846-e54cd46b6eea admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:27.067013 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 835/1683] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:27.085285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dad8093-535c-449c-b846-e54cd46b6eea admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:27.113762 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822968ad-a01d-4b1e-ba82-b6f5c6b92596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:27.114462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822968ad-a01d-4b1e-ba82-b6f5c6b92596 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:27.115512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822968ad-a01d-4b1e-ba82-b6f5c6b92596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:27.115810 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822968ad-a01d-4b1e-ba82-b6f5c6b92596 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:27.116644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822968ad-a01d-4b1e-ba82-b6f5c6b92596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:27.118239 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822968ad-a01d-4b1e-ba82-b6f5c6b92596 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:27.157568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 849/1684] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:27.162073 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2472c-6d27-4f6a-baf1-df7e171f49f4 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:27.178970 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2472c-6d27-4f6a-baf1-df7e171f49f4 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:27.189125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433a1c6-8265-4d88-b765-ee1bdaf512f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:27.190124 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433a1c6-8265-4d88-b765-ee1bdaf512f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:27.190684 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433a1c6-8265-4d88-b765-ee1bdaf512f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:27.191108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433a1c6-8265-4d88-b765-ee1bdaf512f7 admin admin] PATH_INFO: `/v3/projects/bb3948173f294a9381136b4509652fc7/users/64e98a0d50994e19bb2706545dd843d4/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:27.216332 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1433a1c6-8265-4d88-b765-ee1bdaf512f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb3948173f294a9381136b4509652fc7, user_id=64e98a0d50994e19bb2706545dd843d4, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:27.220011 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1433a1c6-8265-4d88-b765-ee1bdaf512f7 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:27.326290 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce7a86-d5b2-40a4-9b1c-1c79dd02f7f2 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:27.326535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce7a86-d5b2-40a4-9b1c-1c79dd02f7f2 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:27.326677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce7a86-d5b2-40a4-9b1c-1c79dd02f7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:27.345117 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 836/1685] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:27 2022] PUT /identity/v3/projects/bb3948173f294a9381136b4509652fc7/users/64e98a0d50994e19bb2706545dd843d4/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:27.356432 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87348d5f-cd11-47d6-a87a-c5470d1f7fdc None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:27.356800 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87348d5f-cd11-47d6-a87a-c5470d1f7fdc None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:27.357166 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87348d5f-cd11-47d6-a87a-c5470d1f7fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:27.417035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-37ce7a86-d5b2-40a4-9b1c-1c79dd02f7f2 None None] MFA Rules not processed for user `50918811a90a444eaf123b2aeb4a75b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:27.419060 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-37ce7a86-d5b2-40a4-9b1c-1c79dd02f7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:27.463900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-87348d5f-cd11-47d6-a87a-c5470d1f7fdc None None] MFA Rules not processed for user `64e98a0d50994e19bb2706545dd843d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:27.465377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-87348d5f-cd11-47d6-a87a-c5470d1f7fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:27.673536 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 850/1686] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:27 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:27.685408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:27.687040 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce7a86-d5b2-40a4-9b1c-1c79dd02f7f2 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:27.692758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-37ce7a86-d5b2-40a4-9b1c-1c79dd02f7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:27.712002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 837/1687] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:27.732665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87348d5f-cd11-47d6-a87a-c5470d1f7fdc None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:27.861694 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87348d5f-cd11-47d6-a87a-c5470d1f7fdc None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:27.869577 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99280fce-0e90-4f11-afc2-21d609528ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:27.871403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99280fce-0e90-4f11-afc2-21d609528ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:27.871904 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99280fce-0e90-4f11-afc2-21d609528ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:27.872332 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99280fce-0e90-4f11-afc2-21d609528ec6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:27.873187 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99280fce-0e90-4f11-afc2-21d609528ec6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:27.875630 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99280fce-0e90-4f11-afc2-21d609528ec6 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:27.928519 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce7a86-d5b2-40a4-9b1c-1c79dd02f7f2 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:27.956687 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe1e33-7136-4bee-be7d-41ed55ff0b13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:27.962032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe1e33-7136-4bee-be7d-41ed55ff0b13 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:27.963887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe1e33-7136-4bee-be7d-41ed55ff0b13 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:27.964815 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe1e33-7136-4bee-be7d-41ed55ff0b13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:27.993710 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe1e33-7136-4bee-be7d-41ed55ff0b13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:27.996593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe1e33-7136-4bee-be7d-41ed55ff0b13 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:28.024697 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 851/1688] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:27 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:28.037540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 838/1689] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:06:27 2022] POST /identity/v3/projects => generated 373 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:28.039118 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0d683-f8b4-4edc-92a5-3a770b18e236 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:28.039118 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0d683-f8b4-4edc-92a5-3a770b18e236 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:28.039118 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc0d683-f8b4-4edc-92a5-3a770b18e236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:28.050541 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a91a6-72f3-4268-8980-73c69337306d None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:28.050838 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a91a6-72f3-4268-8980-73c69337306d None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:28.051119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304a91a6-72f3-4268-8980-73c69337306d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:28.113321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-7dc0d683-f8b4-4edc-92a5-3a770b18e236 None None] MFA Rules not processed for user `806f0c03435e4c5daeadaa4eb437d936`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:28.114943 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7dc0d683-f8b4-4edc-92a5-3a770b18e236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:28.131976 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-304a91a6-72f3-4268-8980-73c69337306d None None] MFA Rules not processed for user `7fd6b148658e4c2884a32f66326833b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:28.133379 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-304a91a6-72f3-4268-8980-73c69337306d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:28.243290 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:28.255294 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 852/1690] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:28.256410 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0d683-f8b4-4edc-92a5-3a770b18e236 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:28.280107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 839/1691] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:28.305555 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4973f-47dc-4672-87ef-692d478e9e37 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:28.306354 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4973f-47dc-4672-87ef-692d478e9e37 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:28.307454 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a4973f-47dc-4672-87ef-692d478e9e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:28.360697 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-f0a4973f-47dc-4672-87ef-692d478e9e37 None None] MFA Rules not processed for user `7fd6b148658e4c2884a32f66326833b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:28.368263 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f0a4973f-47dc-4672-87ef-692d478e9e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:28.405520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 840/1692] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:28 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:06:28.409054 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4973f-47dc-4672-87ef-692d478e9e37 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:28.418831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f0a4973f-47dc-4672-87ef-692d478e9e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:28.449363 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc0d683-f8b4-4edc-92a5-3a770b18e236 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:28.480378 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe724e-5606-42d5-800d-063719586d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:28.482131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe724e-5606-42d5-800d-063719586d2f admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:28.482378 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe724e-5606-42d5-800d-063719586d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:28.482693 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fe724e-5606-42d5-800d-063719586d2f admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:28.483338 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe724e-5606-42d5-800d-063719586d2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:28.485432 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fe724e-5606-42d5-800d-063719586d2f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:28.545158 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a4973f-47dc-4672-87ef-692d478e9e37 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:28.563509 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8815c17-176a-4392-bf79-8d4137eeabb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:28.563775 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8815c17-176a-4392-bf79-8d4137eeabb0 service nova] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:28.564876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8815c17-176a-4392-bf79-8d4137eeabb0 service nova] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:28.565352 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8815c17-176a-4392-bf79-8d4137eeabb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:28.588295 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8815c17-176a-4392-bf79-8d4137eeabb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:28.593075 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8815c17-176a-4392-bf79-8d4137eeabb0 service nova] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:28.594732 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 853/1693] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:06:28 2022] POST /identity/v3/users => generated 293 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:28.595869 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe724e-5606-42d5-800d-063719586d2f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:28.625315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fe724e-5606-42d5-800d-063719586d2f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:28.628219 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 841/1694] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:28.639383 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030b060-883c-4e88-8fa2-2f8849167262 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:28.640193 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030b060-883c-4e88-8fa2-2f8849167262 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:28.640624 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030b060-883c-4e88-8fa2-2f8849167262 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:28.641347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030b060-883c-4e88-8fa2-2f8849167262 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:28.646766 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8815c17-176a-4392-bf79-8d4137eeabb0 service nova] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:28.660750 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030b060-883c-4e88-8fa2-2f8849167262 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:28.662702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030b060-883c-4e88-8fa2-2f8849167262 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:28.685702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 854/1695] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:28 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:28.707224 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8815c17-176a-4392-bf79-8d4137eeabb0 service nova] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:28.725507 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87c929-d609-43d4-9c93-12469aec7137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:28.726169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87c929-d609-43d4-9c93-12469aec7137 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:28.726453 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87c929-d609-43d4-9c93-12469aec7137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:28.726730 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87c929-d609-43d4-9c93-12469aec7137 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:28.727167 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030b060-883c-4e88-8fa2-2f8849167262 service nova] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:28.727429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87c929-d609-43d4-9c93-12469aec7137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:28.728669 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f030b060-883c-4e88-8fa2-2f8849167262 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:28.729068 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87c929-d609-43d4-9c93-12469aec7137 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:28.760558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 842/1696] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:28.763973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87c929-d609-43d4-9c93-12469aec7137 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:28.789965 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87c929-d609-43d4-9c93-12469aec7137 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:28.814812 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ef1c8-deec-4ab9-8685-933a030cdd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:28.818355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ef1c8-deec-4ab9-8685-933a030cdd07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:28.818617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ef1c8-deec-4ab9-8685-933a030cdd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:28.818860 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ef1c8-deec-4ab9-8685-933a030cdd07 admin admin] PATH_INFO: `/v3/projects/de1cd0d023d143d8b46fc9648aacaabc/users/3dd7c185a642400c99ec999eacf859bb/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:28.861513 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:28.875117 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ef1c8-deec-4ab9-8685-933a030cdd07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de1cd0d023d143d8b46fc9648aacaabc, user_id=3dd7c185a642400c99ec999eacf859bb, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:28.878044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ef1c8-deec-4ab9-8685-933a030cdd07 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:28.887982 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030b060-883c-4e88-8fa2-2f8849167262 service nova] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:28.901311 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cce6ba-bbc8-45c2-926e-677901fdea35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:28.902588 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cce6ba-bbc8-45c2-926e-677901fdea35 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:28.902939 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cce6ba-bbc8-45c2-926e-677901fdea35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:28.903104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cce6ba-bbc8-45c2-926e-677901fdea35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:28.916030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e0cce6ba-bbc8-45c2-926e-677901fdea35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:28.979237 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 843/1697] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:28 2022] PUT /identity/v3/projects/de1cd0d023d143d8b46fc9648aacaabc/users/3dd7c185a642400c99ec999eacf859bb/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:06:28.984115 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ef1c8-deec-4ab9-8685-933a030cdd07 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:29.106520 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cce6ba-bbc8-45c2-926e-677901fdea35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:29.108099 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cce6ba-bbc8-45c2-926e-677901fdea35 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:29.114858 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ef1c8-deec-4ab9-8685-933a030cdd07 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:29.126813 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266a243-0eb4-4ce9-b3a2-b943dcdcdaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:29.127489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266a243-0eb4-4ce9-b3a2-b943dcdcdaa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:29.129584 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 855/1698] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:29.130225 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266a243-0eb4-4ce9-b3a2-b943dcdcdaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:29.130697 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266a243-0eb4-4ce9-b3a2-b943dcdcdaa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:29.131769 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266a243-0eb4-4ce9-b3a2-b943dcdcdaa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:29.134073 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266a243-0eb4-4ce9-b3a2-b943dcdcdaa7 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:29.147261 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 844/1699] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:29.151197 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cce6ba-bbc8-45c2-926e-677901fdea35 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:29.169570 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cce6ba-bbc8-45c2-926e-677901fdea35 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:29.179541 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc92ba-0fd7-41a9-bfce-fdeeb4c2f44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:29.181442 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc92ba-0fd7-41a9-bfce-fdeeb4c2f44a admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:29.185844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc92ba-0fd7-41a9-bfce-fdeeb4c2f44a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:29.186255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc92ba-0fd7-41a9-bfce-fdeeb4c2f44a admin admin] PATH_INFO: `/v3/projects/de1cd0d023d143d8b46fc9648aacaabc/users/3dd7c185a642400c99ec999eacf859bb/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:29.193208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc92ba-0fd7-41a9-bfce-fdeeb4c2f44a admin admin] RBAC: Authorizing `identity:create_grant(project_id=de1cd0d023d143d8b46fc9648aacaabc, user_id=3dd7c185a642400c99ec999eacf859bb, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:29.205244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc92ba-0fd7-41a9-bfce-fdeeb4c2f44a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:29.235281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 856/1700] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:29 2022] PUT /identity/v3/projects/de1cd0d023d143d8b46fc9648aacaabc/users/3dd7c185a642400c99ec999eacf859bb/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:29.248796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054a26e-5eac-4141-8b2a-21130be57de8 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:29.249092 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054a26e-5eac-4141-8b2a-21130be57de8 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:29.249380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054a26e-5eac-4141-8b2a-21130be57de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:29.362191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-c054a26e-5eac-4141-8b2a-21130be57de8 None None] MFA Rules not processed for user `3dd7c185a642400c99ec999eacf859bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:29.363967 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-c054a26e-5eac-4141-8b2a-21130be57de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:29.370100 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:29.530458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 845/1701] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:29 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:29.690887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc92ba-0fd7-41a9-bfce-fdeeb4c2f44a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:29.783343 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:29.793645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc92ba-0fd7-41a9-bfce-fdeeb4c2f44a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:29.800925 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657bb28-7843-4b6b-a7b3-85676b2a6de5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:29.802549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657bb28-7843-4b6b-a7b3-85676b2a6de5 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:29.803104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657bb28-7843-4b6b-a7b3-85676b2a6de5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:29.803674 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657bb28-7843-4b6b-a7b3-85676b2a6de5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:29.836574 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657bb28-7843-4b6b-a7b3-85676b2a6de5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:29.852817 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657bb28-7843-4b6b-a7b3-85676b2a6de5 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:29.884014 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 857/1702] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:29 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:30.243939 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054a26e-5eac-4141-8b2a-21130be57de8 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:30.270832 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054a26e-5eac-4141-8b2a-21130be57de8 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:30.285051 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244a2da-7200-4758-ac0e-8c83d0fd4b75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:30.286535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244a2da-7200-4758-ac0e-8c83d0fd4b75 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:30.287154 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244a2da-7200-4758-ac0e-8c83d0fd4b75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:30.288773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6244a2da-7200-4758-ac0e-8c83d0fd4b75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:30.391678 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d831f-c71c-4ae2-8d18-21e90ae226f6 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:30.391914 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d831f-c71c-4ae2-8d18-21e90ae226f6 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:30.392673 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d831f-c71c-4ae2-8d18-21e90ae226f6 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:30.394697 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 858/1703] 10.222.0.46 () {54 vars in 912 bytes} [Tue Aug 2 09:06:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:06:30.409089 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5d4f6-897e-4a72-95c9-0fdbf8c6f2bd None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:30.409502 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5d4f6-897e-4a72-95c9-0fdbf8c6f2bd None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:30.409855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5d4f6-897e-4a72-95c9-0fdbf8c6f2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:30.508082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244a2da-7200-4758-ac0e-8c83d0fd4b75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:30.511752 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6244a2da-7200-4758-ac0e-8c83d0fd4b75 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:30.527420 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-e5c5d4f6-897e-4a72-95c9-0fdbf8c6f2bd None None] MFA Rules not processed for user `ddf6f4d84bb34d2f810a0f6cec36da4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:30.528693 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e5c5d4f6-897e-4a72-95c9-0fdbf8c6f2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:30.537115 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 846/1704] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:30.666964 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 859/1705] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 2 09:06:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:06:30.679015 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244a2da-7200-4758-ac0e-8c83d0fd4b75 service cinder] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:30.686874 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-6244a2da-7200-4758-ac0e-8c83d0fd4b75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:30.720148 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6244a2da-7200-4758-ac0e-8c83d0fd4b75 service cinder] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:30.728089 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbabf6d-14ad-40c4-bdc4-d42b867cf5fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:30.729277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbabf6d-14ad-40c4-bdc4-d42b867cf5fb service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:30.729716 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbabf6d-14ad-40c4-bdc4-d42b867cf5fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:30.730276 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbabf6d-14ad-40c4-bdc4-d42b867cf5fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:30.745496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbabf6d-14ad-40c4-bdc4-d42b867cf5fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:30.747655 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbabf6d-14ad-40c4-bdc4-d42b867cf5fb service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:30.762382 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 847/1706] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:32.002256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f057b25-5874-43d6-9b76-e359704365fa None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:32.002713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f057b25-5874-43d6-9b76-e359704365fa None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:32.003079 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f057b25-5874-43d6-9b76-e359704365fa None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:32.004923 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 860/1707] 10.222.0.46 () {54 vars in 961 bytes} [Tue Aug 2 09:06:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:06:32.021862 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c73472-b374-4096-968c-faa87587bc4b None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:32.022903 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c73472-b374-4096-968c-faa87587bc4b None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:32.023331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c73472-b374-4096-968c-faa87587bc4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:32.078089 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-78c73472-b374-4096-968c-faa87587bc4b None None] MFA Rules not processed for user `cad5cc9927374836b0cc74db85b686fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:32.082989 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-78c73472-b374-4096-968c-faa87587bc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:32.278731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:32.352704 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 848/1708] 10.222.0.46 () {58 vars in 1058 bytes} [Tue Aug 2 09:06:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:32.361480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f057b25-5874-43d6-9b76-e359704365fa None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:32.396143 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f057b25-5874-43d6-9b76-e359704365fa None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:32.428944 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b22dff-7d37-46c9-a329-26efce698b08 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad5cc9927374836b0cc74db85b686fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:32.430289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b22dff-7d37-46c9-a329-26efce698b08 service placement] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:32.430723 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b22dff-7d37-46c9-a329-26efce698b08 service placement] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:32.431144 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b22dff-7d37-46c9-a329-26efce698b08 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:32.434028 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-17b22dff-7d37-46c9-a329-26efce698b08 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:32.460650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b22dff-7d37-46c9-a329-26efce698b08 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:32.464513 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b22dff-7d37-46c9-a329-26efce698b08 service placement] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:32.493411 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 861/1709] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:32.703556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1de357-3d25-490a-b84d-3f76f5566248 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:32.703844 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1de357-3d25-490a-b84d-3f76f5566248 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:32.704119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1de357-3d25-490a-b84d-3f76f5566248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:32.759941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-cf1de357-3d25-490a-b84d-3f76f5566248 None None] MFA Rules not processed for user `7fd6b148658e4c2884a32f66326833b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:32.762520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-cf1de357-3d25-490a-b84d-3f76f5566248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:32.976590 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 849/1710] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:06:33.391418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1352d65-1239-479f-9765-e95b308bd696 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:33.392456 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1352d65-1239-479f-9765-e95b308bd696 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:33.392949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1352d65-1239-479f-9765-e95b308bd696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:33.442333 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-c1352d65-1239-479f-9765-e95b308bd696 None None] MFA Rules not processed for user `50918811a90a444eaf123b2aeb4a75b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:33.444339 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c1352d65-1239-479f-9765-e95b308bd696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:33.546991 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 862/1711] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:33.655577 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04dca3-13c4-4f20-9d50-8feef0c474df None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:33.656511 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04dca3-13c4-4f20-9d50-8feef0c474df None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:33.656942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04dca3-13c4-4f20-9d50-8feef0c474df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:33.770244 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-0c04dca3-13c4-4f20-9d50-8feef0c474df None None] MFA Rules not processed for user `3b9d9cfdcaac473786d3017f73163274`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:33.783700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-0c04dca3-13c4-4f20-9d50-8feef0c474df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:34.001181 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 850/1712] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:06:34.013653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1201e4-58d3-4756-b76b-dce3e7b73bd6 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:34.014267 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1201e4-58d3-4756-b76b-dce3e7b73bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:34.014796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1201e4-58d3-4756-b76b-dce3e7b73bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:34.041125 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-4e1201e4-58d3-4756-b76b-dce3e7b73bd6 None None] MFA Rules not processed for user `3b9d9cfdcaac473786d3017f73163274`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:34.044039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4e1201e4-58d3-4756-b76b-dce3e7b73bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:34.068897 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 863/1713] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:34.720431 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31b3f3-4624-46a0-91ef-a0e725112a0e None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:34.721610 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31b3f3-4624-46a0-91ef-a0e725112a0e None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:34.722142 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31b3f3-4624-46a0-91ef-a0e725112a0e None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:34.723778 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 851/1714] 10.222.0.46 () {54 vars in 912 bytes} [Tue Aug 2 09:06:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 02 09:06:34.730498 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fe696-2999-4623-b872-bcb68afe6790 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:34.731433 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fe696-2999-4623-b872-bcb68afe6790 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:34.731985 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124fe696-2999-4623-b872-bcb68afe6790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:34.775616 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-124fe696-2999-4623-b872-bcb68afe6790 None None] MFA Rules not processed for user `ddf6f4d84bb34d2f810a0f6cec36da4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:34.796817 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-124fe696-2999-4623-b872-bcb68afe6790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:34.818012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 864/1715] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 2 09:06:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:34.860019 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31b3f3-4624-46a0-91ef-a0e725112a0e None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:34.862055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-5e31b3f3-4624-46a0-91ef-a0e725112a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:34.881276 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31b3f3-4624-46a0-91ef-a0e725112a0e None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:34.886979 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90458c38-0e8e-4a22-9329-208b0badad83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:34.887427 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90458c38-0e8e-4a22-9329-208b0badad83 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:34.887716 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90458c38-0e8e-4a22-9329-208b0badad83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:34.887996 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90458c38-0e8e-4a22-9329-208b0badad83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:34.900727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90458c38-0e8e-4a22-9329-208b0badad83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:34.904101 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90458c38-0e8e-4a22-9329-208b0badad83 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:34.919417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 852/1716] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:35.478674 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547eaa9a-3b29-4214-8b88-8d259d4d096c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:35.479257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547eaa9a-3b29-4214-8b88-8d259d4d096c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:35.479651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547eaa9a-3b29-4214-8b88-8d259d4d096c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:35.514341 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-547eaa9a-3b29-4214-8b88-8d259d4d096c None None] MFA Rules not processed for user `806f0c03435e4c5daeadaa4eb437d936`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:35.517655 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-547eaa9a-3b29-4214-8b88-8d259d4d096c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:35.653207 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 865/1717] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:35.676796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90458c38-0e8e-4a22-9329-208b0badad83 service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:35.878795 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90458c38-0e8e-4a22-9329-208b0badad83 service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:35.897222 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffb8a69-e9d8-4635-996c-a5c227243855 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:35.897871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb8a69-e9d8-4635-996c-a5c227243855 service nova] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:35.898175 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb8a69-e9d8-4635-996c-a5c227243855 service nova] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:35.899553 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffb8a69-e9d8-4635-996c-a5c227243855 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:35.953017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:35.961060 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb8a69-e9d8-4635-996c-a5c227243855 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:35.962951 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffb8a69-e9d8-4635-996c-a5c227243855 service nova] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:35.982908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 853/1718] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:38.400252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547eaa9a-3b29-4214-8b88-8d259d4d096c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:38.404660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-547eaa9a-3b29-4214-8b88-8d259d4d096c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:38.442539 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:38.656065 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547eaa9a-3b29-4214-8b88-8d259d4d096c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:38.662353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681be457-af18-4e3d-acc5-3befbd3b7311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:38.664745 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681be457-af18-4e3d-acc5-3befbd3b7311 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:38.666073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681be457-af18-4e3d-acc5-3befbd3b7311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:38.666564 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681be457-af18-4e3d-acc5-3befbd3b7311 admin admin] PATH_INFO: `/v3/users/87af3620e39a4009905152fcda5a8e23` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:38.669283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681be457-af18-4e3d-acc5-3befbd3b7311 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87af3620e39a4009905152fcda5a8e23)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:38.673146 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681be457-af18-4e3d-acc5-3befbd3b7311 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:38.817320 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204c7ac-5746-4691-924d-47bcba84af43 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:38.817854 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204c7ac-5746-4691-924d-47bcba84af43 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:38.817854 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204c7ac-5746-4691-924d-47bcba84af43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:39.059772 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-681be457-af18-4e3d-acc5-3befbd3b7311 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87af3620e39a4009905152fcda5a8e23'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:39.069863 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-0204c7ac-5746-4691-924d-47bcba84af43 None None] MFA Rules not processed for user `ed8114b7a28741268fdff71c866dbc77`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:39.072357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-0204c7ac-5746-4691-924d-47bcba84af43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:39.082353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-681be457-af18-4e3d-acc5-3befbd3b7311 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87af3620e39a4009905152fcda5a8e23'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:39.083318 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-681be457-af18-4e3d-acc5-3befbd3b7311 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87af3620e39a4009905152fcda5a8e23'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:39.128708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-681be457-af18-4e3d-acc5-3befbd3b7311 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87af3620e39a4009905152fcda5a8e23'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:39.129392 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-681be457-af18-4e3d-acc5-3befbd3b7311 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87af3620e39a4009905152fcda5a8e23'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:39.303515 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 854/1719] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 2 09:06:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 487 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:06:39.456784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 866/1720] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:06:38 2022] DELETE /identity/v3/users/87af3620e39a4009905152fcda5a8e23 => generated 0 bytes in 1058 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:39.461336 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3d4a1-89ca-49d6-bac0-e5b2337275c6 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:39.461759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3d4a1-89ca-49d6-bac0-e5b2337275c6 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:39.462160 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3d4a1-89ca-49d6-bac0-e5b2337275c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:39.482231 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-22c3d4a1-89ca-49d6-bac0-e5b2337275c6 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:39.486520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-22c3d4a1-89ca-49d6-bac0-e5b2337275c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:39.553513 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 855/1721] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:39.564908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681be457-af18-4e3d-acc5-3befbd3b7311 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:39.566359 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-681be457-af18-4e3d-acc5-3befbd3b7311 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:39.626862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681be457-af18-4e3d-acc5-3befbd3b7311 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:39.631242 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30fd55-6ee1-49cf-a554-5c707566ff6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:39.631909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30fd55-6ee1-49cf-a554-5c707566ff6d service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:39.632223 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30fd55-6ee1-49cf-a554-5c707566ff6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:39.632501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f30fd55-6ee1-49cf-a554-5c707566ff6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:39.649966 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30fd55-6ee1-49cf-a554-5c707566ff6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:39.649966 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f30fd55-6ee1-49cf-a554-5c707566ff6d service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:39.658727 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 867/1722] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:39.820610 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3d4a1-89ca-49d6-bac0-e5b2337275c6 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:39.821846 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-22c3d4a1-89ca-49d6-bac0-e5b2337275c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:39.929097 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:39.959827 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3d4a1-89ca-49d6-bac0-e5b2337275c6 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:39.965818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b734cd2-3a81-413c-8e11-778eeba290ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff92ef3b9da74ccab3fbc43f852e3ad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:39.966578 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b734cd2-3a81-413c-8e11-778eeba290ec service swift] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:39.966930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b734cd2-3a81-413c-8e11-778eeba290ec service swift] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:39.967273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b734cd2-3a81-413c-8e11-778eeba290ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:40.029940 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b734cd2-3a81-413c-8e11-778eeba290ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:40.036266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b734cd2-3a81-413c-8e11-778eeba290ec service swift] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:40.045530 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 856/1723] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 2 09:06:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 02 09:06:40.538947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30fd55-6ee1-49cf-a554-5c707566ff6d service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:40.540563 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6f30fd55-6ee1-49cf-a554-5c707566ff6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:40.559466 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f30fd55-6ee1-49cf-a554-5c707566ff6d service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:40.566234 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d15c82-741f-4e71-ac83-19e7ef77bbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:40.566769 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d15c82-741f-4e71-ac83-19e7ef77bbb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:40.567054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d15c82-741f-4e71-ac83-19e7ef77bbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:40.567239 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d15c82-741f-4e71-ac83-19e7ef77bbb2 admin admin] PATH_INFO: `/v3/projects/edc8f82022af4e398c2c0fbfe555ef88` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:40.568716 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d15c82-741f-4e71-ac83-19e7ef77bbb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edc8f82022af4e398c2c0fbfe555ef88)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:40.570695 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d15c82-741f-4e71-ac83-19e7ef77bbb2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:40.709159 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:40.728708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-58d15c82-741f-4e71-ac83-19e7ef77bbb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edc8f82022af4e398c2c0fbfe555ef88', 'request_id': 'req-58d15c82-741f-4e71-ac83-19e7ef77bbb2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:40.735353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-58d15c82-741f-4e71-ac83-19e7ef77bbb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edc8f82022af4e398c2c0fbfe555ef88', 'request_id': 'req-58d15c82-741f-4e71-ac83-19e7ef77bbb2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:40.815229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-58d15c82-741f-4e71-ac83-19e7ef77bbb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edc8f82022af4e398c2c0fbfe555ef88', 'request_id': 'req-58d15c82-741f-4e71-ac83-19e7ef77bbb2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:40.836504 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-58d15c82-741f-4e71-ac83-19e7ef77bbb2 admin admin] The token cache is being invalidate because project edc8f82022af4e398c2c0fbfe555ef88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:06:40.836859 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-58d15c82-741f-4e71-ac83-19e7ef77bbb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:40.837561 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-58d15c82-741f-4e71-ac83-19e7ef77bbb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:40.840148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 868/1724] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:06:40 2022] DELETE /identity/v3/projects/edc8f82022af4e398c2c0fbfe555ef88 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:40.845286 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da3ca9-d8a0-4697-b6e1-92863e569331 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:40.847306 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da3ca9-d8a0-4697-b6e1-92863e569331 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:40.847306 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da3ca9-d8a0-4697-b6e1-92863e569331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:40.867765 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-b0da3ca9-d8a0-4697-b6e1-92863e569331 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:40.868989 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-b0da3ca9-d8a0-4697-b6e1-92863e569331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:40.960261 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 857/1725] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:40.969106 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e7900-0189-4c3d-9d62-3e61a3d74231 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:40.969395 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e7900-0189-4c3d-9d62-3e61a3d74231 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:40.969675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e7900-0189-4c3d-9d62-3e61a3d74231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:40.991563 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9f8e7900-0189-4c3d-9d62-3e61a3d74231 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:40.992910 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9f8e7900-0189-4c3d-9d62-3e61a3d74231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:41.001086 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 869/1726] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:41.005543 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da3ca9-d8a0-4697-b6e1-92863e569331 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:41.015678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da3ca9-d8a0-4697-b6e1-92863e569331 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:41.020458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05920897-1765-43e6-94c3-1a78c8a18a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:41.021074 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05920897-1765-43e6-94c3-1a78c8a18a67 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:41.021365 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05920897-1765-43e6-94c3-1a78c8a18a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:41.021644 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05920897-1765-43e6-94c3-1a78c8a18a67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:41.022385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05920897-1765-43e6-94c3-1a78c8a18a67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:06:41.022695 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05920897-1765-43e6-94c3-1a78c8a18a67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:41.024290 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05920897-1765-43e6-94c3-1a78c8a18a67 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:41.037024 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 858/1727] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:06:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:06:41.041605 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e7900-0189-4c3d-9d62-3e61a3d74231 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:41.052039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e7900-0189-4c3d-9d62-3e61a3d74231 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:41.056680 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83653103-637b-4361-ac93-d5d375864af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:41.057293 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83653103-637b-4361-ac93-d5d375864af9 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:41.057575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83653103-637b-4361-ac93-d5d375864af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:41.057848 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83653103-637b-4361-ac93-d5d375864af9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:41.058542 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83653103-637b-4361-ac93-d5d375864af9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:41.060507 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83653103-637b-4361-ac93-d5d375864af9 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:41.137430 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 870/1728] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:06:41 2022] POST /identity/v3/projects => generated 373 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:41.141652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05920897-1765-43e6-94c3-1a78c8a18a67 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:41.152168 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05920897-1765-43e6-94c3-1a78c8a18a67 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:41.217168 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ceba6e-8a1f-4038-8080-23fc7b105b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:41.217803 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ceba6e-8a1f-4038-8080-23fc7b105b45 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:41.218143 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ceba6e-8a1f-4038-8080-23fc7b105b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:41.218422 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ceba6e-8a1f-4038-8080-23fc7b105b45 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:41.219108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ceba6e-8a1f-4038-8080-23fc7b105b45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:41.221095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ceba6e-8a1f-4038-8080-23fc7b105b45 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:41.332100 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 859/1729] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:06:41 2022] POST /identity/v3/users => generated 287 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:06:41.336328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83653103-637b-4361-ac93-d5d375864af9 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:41.347380 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83653103-637b-4361-ac93-d5d375864af9 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:41.352756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01779d25-44d6-4761-942a-8352bd55abe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:41.353605 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01779d25-44d6-4761-942a-8352bd55abe2 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:41.354007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01779d25-44d6-4761-942a-8352bd55abe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:41.354935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01779d25-44d6-4761-942a-8352bd55abe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:41.355735 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01779d25-44d6-4761-942a-8352bd55abe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:41.357395 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01779d25-44d6-4761-942a-8352bd55abe2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:41.368503 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 871/1730] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:41.373521 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ceba6e-8a1f-4038-8080-23fc7b105b45 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:41.384463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ceba6e-8a1f-4038-8080-23fc7b105b45 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:41.389314 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d3613-befd-4d43-b485-42009cae6526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:41.389985 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d3613-befd-4d43-b485-42009cae6526 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:41.390451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d3613-befd-4d43-b485-42009cae6526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:41.390582 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d3613-befd-4d43-b485-42009cae6526 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:41.391255 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8d3613-befd-4d43-b485-42009cae6526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:41.392954 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8d3613-befd-4d43-b485-42009cae6526 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:41.394539 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:41.411193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 860/1731] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:41.415430 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01779d25-44d6-4761-942a-8352bd55abe2 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:41.427605 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01779d25-44d6-4761-942a-8352bd55abe2 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:41.433145 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4da5d8-3a49-4763-9808-e3c815532444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:41.434142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4da5d8-3a49-4763-9808-e3c815532444 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:41.434498 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4da5d8-3a49-4763-9808-e3c815532444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:41.434845 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4da5d8-3a49-4763-9808-e3c815532444 admin admin] PATH_INFO: `/v3/projects/51ec54c441964d8baba1393e3c2391ca/users/41c2a86dec694ff78703b99bff6c2ef2/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:41.453342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4da5d8-3a49-4763-9808-e3c815532444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51ec54c441964d8baba1393e3c2391ca, user_id=41c2a86dec694ff78703b99bff6c2ef2, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:41.456821 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4da5d8-3a49-4763-9808-e3c815532444 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:41.538189 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 872/1732] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:41 2022] PUT /identity/v3/projects/51ec54c441964d8baba1393e3c2391ca/users/41c2a86dec694ff78703b99bff6c2ef2/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:41.551857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bf34d-53e7-46b8-bd57-71aaa2414ed9 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:41.552626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bf34d-53e7-46b8-bd57-71aaa2414ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:41.552989 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bf34d-53e7-46b8-bd57-71aaa2414ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:41.599997 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-b84bf34d-53e7-46b8-bd57-71aaa2414ed9 None None] MFA Rules not processed for user `41c2a86dec694ff78703b99bff6c2ef2`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:41.601433 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-b84bf34d-53e7-46b8-bd57-71aaa2414ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:41.688207 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 861/1733] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:41.693023 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4da5d8-3a49-4763-9808-e3c815532444 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:41.775577 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4da5d8-3a49-4763-9808-e3c815532444 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:41.781189 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1464e9-ed25-4e6f-8039-01c78f10e48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:41.781854 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1464e9-ed25-4e6f-8039-01c78f10e48a admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:41.782395 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1464e9-ed25-4e6f-8039-01c78f10e48a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:41.782683 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1464e9-ed25-4e6f-8039-01c78f10e48a admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:41.783386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1464e9-ed25-4e6f-8039-01c78f10e48a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:41.785362 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1464e9-ed25-4e6f-8039-01c78f10e48a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:41.882792 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 873/1734] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:06:41 2022] POST /identity/v3/projects => generated 371 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:41.883843 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84bf34d-53e7-46b8-bd57-71aaa2414ed9 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:41.897546 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84bf34d-53e7-46b8-bd57-71aaa2414ed9 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:42.028797 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9dd0a9-d5a2-41ac-95f3-086d9b2fc910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:42.029521 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9dd0a9-d5a2-41ac-95f3-086d9b2fc910 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:42.034446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9dd0a9-d5a2-41ac-95f3-086d9b2fc910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:42.034994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9dd0a9-d5a2-41ac-95f3-086d9b2fc910 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:42.036157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9dd0a9-d5a2-41ac-95f3-086d9b2fc910 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:42.038458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9dd0a9-d5a2-41ac-95f3-086d9b2fc910 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:42.165930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 862/1735] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:06:41 2022] POST /identity/v3/users => generated 292 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:06:42.170443 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1464e9-ed25-4e6f-8039-01c78f10e48a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:42.184124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1464e9-ed25-4e6f-8039-01c78f10e48a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:42.190143 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6f2d9-af72-46de-b948-3319389cda87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:42.191713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6f2d9-af72-46de-b948-3319389cda87 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:42.191713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6f2d9-af72-46de-b948-3319389cda87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:42.191713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d6f2d9-af72-46de-b948-3319389cda87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:42.192415 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d6f2d9-af72-46de-b948-3319389cda87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:42.194086 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d6f2d9-af72-46de-b948-3319389cda87 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:42.211125 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 874/1736] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:42.212348 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9dd0a9-d5a2-41ac-95f3-086d9b2fc910 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:42.223320 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9dd0a9-d5a2-41ac-95f3-086d9b2fc910 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:42.228713 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0846642-e8b4-4bb7-8483-f07cfbf133d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:42.230082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0846642-e8b4-4bb7-8483-f07cfbf133d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:42.230634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0846642-e8b4-4bb7-8483-f07cfbf133d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:42.230844 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0846642-e8b4-4bb7-8483-f07cfbf133d3 admin admin] PATH_INFO: `/v3/projects/b18d73a0533f43889ddc02e840309d4c/users/11d82532d7bb4056953d9f78120be911/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:42.258347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0846642-e8b4-4bb7-8483-f07cfbf133d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b18d73a0533f43889ddc02e840309d4c, user_id=11d82532d7bb4056953d9f78120be911, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:42.261117 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0846642-e8b4-4bb7-8483-f07cfbf133d3 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:42.413496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 863/1737] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:42 2022] PUT /identity/v3/projects/b18d73a0533f43889ddc02e840309d4c/users/11d82532d7bb4056953d9f78120be911/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:06:42.418386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6f2d9-af72-46de-b948-3319389cda87 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:42.435574 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d6f2d9-af72-46de-b948-3319389cda87 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:42.464752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:42.519777 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ede7c-e213-4368-8edd-62d9e9c947e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:42.520431 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ede7c-e213-4368-8edd-62d9e9c947e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:42.520718 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ede7c-e213-4368-8edd-62d9e9c947e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:42.520991 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ede7c-e213-4368-8edd-62d9e9c947e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:42.521730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ede7c-e213-4368-8edd-62d9e9c947e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:42.523355 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ede7c-e213-4368-8edd-62d9e9c947e0 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:42.537748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 875/1738] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:06:42.539786 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0846642-e8b4-4bb7-8483-f07cfbf133d3 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:42.847026 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0846642-e8b4-4bb7-8483-f07cfbf133d3 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:42.847026 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a55ad-16bf-4a16-a6cf-912129021210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:42.847026 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a55ad-16bf-4a16-a6cf-912129021210 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:42.847026 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a55ad-16bf-4a16-a6cf-912129021210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:42.849283 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a55ad-16bf-4a16-a6cf-912129021210 admin admin] PATH_INFO: `/v3/projects/b18d73a0533f43889ddc02e840309d4c/users/11d82532d7bb4056953d9f78120be911/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:42.849283 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a55ad-16bf-4a16-a6cf-912129021210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b18d73a0533f43889ddc02e840309d4c, user_id=11d82532d7bb4056953d9f78120be911, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:42.849283 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a55ad-16bf-4a16-a6cf-912129021210 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:42.849283 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 864/1739] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:42 2022] PUT /identity/v3/projects/b18d73a0533f43889ddc02e840309d4c/users/11d82532d7bb4056953d9f78120be911/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:42.850565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61029098-f35c-4c65-9be2-da50fca84b38 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:42.850565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61029098-f35c-4c65-9be2-da50fca84b38 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:42.850565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61029098-f35c-4c65-9be2-da50fca84b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:42.850565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-61029098-f35c-4c65-9be2-da50fca84b38 None None] MFA Rules not processed for user `11d82532d7bb4056953d9f78120be911`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:42.850565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-61029098-f35c-4c65-9be2-da50fca84b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:42.850565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 876/1740] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:42 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:42.852614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a55ad-16bf-4a16-a6cf-912129021210 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:42.852614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-1f1a55ad-16bf-4a16-a6cf-912129021210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:42.926441 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:42.951195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a55ad-16bf-4a16-a6cf-912129021210 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:42.955396 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd06c2-c575-4749-881a-73784ffcda7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:42.955926 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd06c2-c575-4749-881a-73784ffcda7a service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:42.956199 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd06c2-c575-4749-881a-73784ffcda7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:42.956526 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fd06c2-c575-4749-881a-73784ffcda7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:42.968774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd06c2-c575-4749-881a-73784ffcda7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:42.971300 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fd06c2-c575-4749-881a-73784ffcda7a service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:42.985367 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 865/1741] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:42 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:44.070772 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438391cd-2a78-4aaa-8732-229511dcf3b3 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:44.072008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438391cd-2a78-4aaa-8732-229511dcf3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:44.072537 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438391cd-2a78-4aaa-8732-229511dcf3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:44.107078 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-438391cd-2a78-4aaa-8732-229511dcf3b3 None None] MFA Rules not processed for user `ddf6f4d84bb34d2f810a0f6cec36da4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:44.109770 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-438391cd-2a78-4aaa-8732-229511dcf3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:44.188252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 877/1742] 10.222.0.72 () {58 vars in 1015 bytes} [Tue Aug 2 09:06:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:44.910038 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd06c2-c575-4749-881a-73784ffcda7a service cinder] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:45.020987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fd06c2-c575-4749-881a-73784ffcda7a service cinder] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:45.026682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d70bd-c3fc-4628-9f40-9cb2990f5143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:45.027665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d70bd-c3fc-4628-9f40-9cb2990f5143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:45.027783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d70bd-c3fc-4628-9f40-9cb2990f5143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:45.028326 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01d70bd-c3fc-4628-9f40-9cb2990f5143 admin admin] PATH_INFO: `/v3/users/41c2a86dec694ff78703b99bff6c2ef2` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:45.029741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d70bd-c3fc-4628-9f40-9cb2990f5143 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41c2a86dec694ff78703b99bff6c2ef2)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:45.032380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01d70bd-c3fc-4628-9f40-9cb2990f5143 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:45.323324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-a01d70bd-c3fc-4628-9f40-9cb2990f5143 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41c2a86dec694ff78703b99bff6c2ef2'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:45.331638 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-a01d70bd-c3fc-4628-9f40-9cb2990f5143 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41c2a86dec694ff78703b99bff6c2ef2'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:45.332418 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-a01d70bd-c3fc-4628-9f40-9cb2990f5143 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41c2a86dec694ff78703b99bff6c2ef2'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:45.377147 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438391cd-2a78-4aaa-8732-229511dcf3b3 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:45.394522 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438391cd-2a78-4aaa-8732-229511dcf3b3 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:45.403810 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de81da2-ab13-4119-8864-16cdab46e2f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:45.404918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de81da2-ab13-4119-8864-16cdab46e2f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:45.405252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de81da2-ab13-4119-8864-16cdab46e2f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:45.405643 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de81da2-ab13-4119-8864-16cdab46e2f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:45.408374 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5de81da2-ab13-4119-8864-16cdab46e2f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:45.416298 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-a01d70bd-c3fc-4628-9f40-9cb2990f5143 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41c2a86dec694ff78703b99bff6c2ef2'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:45.434410 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-a01d70bd-c3fc-4628-9f40-9cb2990f5143 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41c2a86dec694ff78703b99bff6c2ef2'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:45.455644 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 866/1743] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:06:44 2022] DELETE /identity/v3/users/41c2a86dec694ff78703b99bff6c2ef2 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:45.457634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d70bd-c3fc-4628-9f40-9cb2990f5143 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:45.460375 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-a01d70bd-c3fc-4628-9f40-9cb2990f5143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:45.490649 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:45.495909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de81da2-ab13-4119-8864-16cdab46e2f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:45.498089 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de81da2-ab13-4119-8864-16cdab46e2f4 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:45.525104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 878/1744] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:06:45.529217 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72642fd-454f-4f0f-aea5-a4f842fa761e None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:45.529492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72642fd-454f-4f0f-aea5-a4f842fa761e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:45.529783 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72642fd-454f-4f0f-aea5-a4f842fa761e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:45.561286 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-c72642fd-454f-4f0f-aea5-a4f842fa761e None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:45.563000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c72642fd-454f-4f0f-aea5-a4f842fa761e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:45.598076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01d70bd-c3fc-4628-9f40-9cb2990f5143 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:45.602483 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf0c3a-e9a6-4612-ba0f-d884010302ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:45.603410 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf0c3a-e9a6-4612-ba0f-d884010302ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:45.603883 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf0c3a-e9a6-4612-ba0f-d884010302ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:45.604324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf0c3a-e9a6-4612-ba0f-d884010302ec admin admin] PATH_INFO: `/v3/users/5e2351a44ccf4f85b2d9b9ab92657e56` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:45.605951 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf0c3a-e9a6-4612-ba0f-d884010302ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e2351a44ccf4f85b2d9b9ab92657e56)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:45.608577 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf0c3a-e9a6-4612-ba0f-d884010302ec admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:45.613338 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:45.658356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 879/1745] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:45.665600 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72642fd-454f-4f0f-aea5-a4f842fa761e None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:45.667907 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c72642fd-454f-4f0f-aea5-a4f842fa761e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:45.810705 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72642fd-454f-4f0f-aea5-a4f842fa761e None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:45.815430 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac35e928-d897-4370-b89b-d59bb189d4d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:45.816068 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35e928-d897-4370-b89b-d59bb189d4d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:45.816352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35e928-d897-4370-b89b-d59bb189d4d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:45.816631 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35e928-d897-4370-b89b-d59bb189d4d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:45.829633 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac35e928-d897-4370-b89b-d59bb189d4d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:45.831157 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac35e928-d897-4370-b89b-d59bb189d4d5 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:45.842804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 880/1746] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:06:45.843564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9bdf0c3a-e9a6-4612-ba0f-d884010302ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e2351a44ccf4f85b2d9b9ab92657e56'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:45.852431 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9bdf0c3a-e9a6-4612-ba0f-d884010302ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e2351a44ccf4f85b2d9b9ab92657e56'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:45.898620 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9bdf0c3a-e9a6-4612-ba0f-d884010302ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e2351a44ccf4f85b2d9b9ab92657e56'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:46.016550 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9bdf0c3a-e9a6-4612-ba0f-d884010302ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e2351a44ccf4f85b2d9b9ab92657e56'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:46.044361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9bdf0c3a-e9a6-4612-ba0f-d884010302ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e2351a44ccf4f85b2d9b9ab92657e56'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:46.051507 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 867/1747] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:06:45 2022] DELETE /identity/v3/users/5e2351a44ccf4f85b2d9b9ab92657e56 => generated 0 bytes in 595 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:46.057014 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08506f-584e-445d-a0d6-658b2b83dab1 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:46.057344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08506f-584e-445d-a0d6-658b2b83dab1 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:46.057642 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f08506f-584e-445d-a0d6-658b2b83dab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:46.081612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9f08506f-584e-445d-a0d6-658b2b83dab1 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:46.083154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9f08506f-584e-445d-a0d6-658b2b83dab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:46.184465 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 881/1748] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:46.199586 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf0c3a-e9a6-4612-ba0f-d884010302ec admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:46.201493 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-9bdf0c3a-e9a6-4612-ba0f-d884010302ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:46.344283 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf0c3a-e9a6-4612-ba0f-d884010302ec admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:46.362073 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a9ff7e-5ff5-4c91-b9ef-e71e20a7a789 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:46.362882 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9ff7e-5ff5-4c91-b9ef-e71e20a7a789 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:46.363232 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9ff7e-5ff5-4c91-b9ef-e71e20a7a789 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:46.363621 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9ff7e-5ff5-4c91-b9ef-e71e20a7a789 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:46.399216 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08506f-584e-445d-a0d6-658b2b83dab1 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:46.399807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a9ff7e-5ff5-4c91-b9ef-e71e20a7a789 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:46.400804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9f08506f-584e-445d-a0d6-658b2b83dab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:46.401319 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a9ff7e-5ff5-4c91-b9ef-e71e20a7a789 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:46.422954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f08506f-584e-445d-a0d6-658b2b83dab1 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:46.436502 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 868/1749] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:46.439374 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3371297-a729-4d22-92ab-f28754e87c53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:46.440060 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3371297-a729-4d22-92ab-f28754e87c53 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:46.440339 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3371297-a729-4d22-92ab-f28754e87c53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:46.440612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3371297-a729-4d22-92ab-f28754e87c53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:46.451193 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d3371297-a729-4d22-92ab-f28754e87c53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:46.479757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3371297-a729-4d22-92ab-f28754e87c53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:46.483567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3371297-a729-4d22-92ab-f28754e87c53 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:46.509980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 882/1750] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:06:46.676968 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a9ff7e-5ff5-4c91-b9ef-e71e20a7a789 service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:46.678782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-71a9ff7e-5ff5-4c91-b9ef-e71e20a7a789 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:46.704059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a9ff7e-5ff5-4c91-b9ef-e71e20a7a789 service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:46.720888 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f6328-4ea5-4995-b912-d9d1454372d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:46.721823 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f6328-4ea5-4995-b912-d9d1454372d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:46.722295 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f6328-4ea5-4995-b912-d9d1454372d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:46.729718 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f6328-4ea5-4995-b912-d9d1454372d4 admin admin] PATH_INFO: `/v3/projects/51ec54c441964d8baba1393e3c2391ca` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:46.734747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0f6328-4ea5-4995-b912-d9d1454372d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51ec54c441964d8baba1393e3c2391ca)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:46.738616 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0f6328-4ea5-4995-b912-d9d1454372d4 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:46.951085 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-af0f6328-4ea5-4995-b912-d9d1454372d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51ec54c441964d8baba1393e3c2391ca', 'request_id': 'req-af0f6328-4ea5-4995-b912-d9d1454372d4'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:46.973083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-af0f6328-4ea5-4995-b912-d9d1454372d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51ec54c441964d8baba1393e3c2391ca', 'request_id': 'req-af0f6328-4ea5-4995-b912-d9d1454372d4'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:46.989339 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-af0f6328-4ea5-4995-b912-d9d1454372d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51ec54c441964d8baba1393e3c2391ca', 'request_id': 'req-af0f6328-4ea5-4995-b912-d9d1454372d4'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:47.102194 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-af0f6328-4ea5-4995-b912-d9d1454372d4 admin admin] The token cache is being invalidate because project 51ec54c441964d8baba1393e3c2391ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:06:47.103183 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-af0f6328-4ea5-4995-b912-d9d1454372d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:47.108122 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-af0f6328-4ea5-4995-b912-d9d1454372d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:47.120556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 869/1751] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:06:46 2022] DELETE /identity/v3/projects/51ec54c441964d8baba1393e3c2391ca => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:47.125787 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3371297-a729-4d22-92ab-f28754e87c53 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:47.197788 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3371297-a729-4d22-92ab-f28754e87c53 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:47.301012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:47.421942 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac9df6-c3e2-4ab0-b361-77a140e17e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:47.423471 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac9df6-c3e2-4ab0-b361-77a140e17e72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:47.423931 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac9df6-c3e2-4ab0-b361-77a140e17e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:47.424367 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac9df6-c3e2-4ab0-b361-77a140e17e72 admin admin] PATH_INFO: `/v3/users/11d82532d7bb4056953d9f78120be911` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:47.426059 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac9df6-c3e2-4ab0-b361-77a140e17e72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11d82532d7bb4056953d9f78120be911)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:47.433560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac9df6-c3e2-4ab0-b361-77a140e17e72 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:47.775663 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-acac9df6-c3e2-4ab0-b361-77a140e17e72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11d82532d7bb4056953d9f78120be911'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:47.789915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-acac9df6-c3e2-4ab0-b361-77a140e17e72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11d82532d7bb4056953d9f78120be911'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:47.790887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-acac9df6-c3e2-4ab0-b361-77a140e17e72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11d82532d7bb4056953d9f78120be911'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:47.796585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f6328-4ea5-4995-b912-d9d1454372d4 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:47.798992 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-af0f6328-4ea5-4995-b912-d9d1454372d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:47.813305 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-acac9df6-c3e2-4ab0-b361-77a140e17e72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11d82532d7bb4056953d9f78120be911'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:47.814305 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-acac9df6-c3e2-4ab0-b361-77a140e17e72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11d82532d7bb4056953d9f78120be911'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:47.836467 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:47.851593 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f6328-4ea5-4995-b912-d9d1454372d4 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:47.865371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df84bfc8-a9e4-4d2b-b1ea-40daab446441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:47.866200 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84bfc8-a9e4-4d2b-b1ea-40daab446441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:47.866625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84bfc8-a9e4-4d2b-b1ea-40daab446441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:47.867081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df84bfc8-a9e4-4d2b-b1ea-40daab446441 admin admin] PATH_INFO: `/v3/projects/4f00d6f249f94245a6e012c3740ff646` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:47.868552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df84bfc8-a9e4-4d2b-b1ea-40daab446441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f00d6f249f94245a6e012c3740ff646)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:47.875733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df84bfc8-a9e4-4d2b-b1ea-40daab446441 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:47.961925 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 883/1752] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:06:47 2022] DELETE /identity/v3/users/11d82532d7bb4056953d9f78120be911 => generated 0 bytes in 836 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:48.076564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-df84bfc8-a9e4-4d2b-b1ea-40daab446441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f00d6f249f94245a6e012c3740ff646', 'request_id': 'req-df84bfc8-a9e4-4d2b-b1ea-40daab446441'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:48.133620 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-df84bfc8-a9e4-4d2b-b1ea-40daab446441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f00d6f249f94245a6e012c3740ff646', 'request_id': 'req-df84bfc8-a9e4-4d2b-b1ea-40daab446441'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:48.168596 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-df84bfc8-a9e4-4d2b-b1ea-40daab446441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f00d6f249f94245a6e012c3740ff646', 'request_id': 'req-df84bfc8-a9e4-4d2b-b1ea-40daab446441'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:48.282372 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-df84bfc8-a9e4-4d2b-b1ea-40daab446441 admin admin] The token cache is being invalidate because project 4f00d6f249f94245a6e012c3740ff646 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:06:48.283162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-df84bfc8-a9e4-4d2b-b1ea-40daab446441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:48.284795 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-df84bfc8-a9e4-4d2b-b1ea-40daab446441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:48.298803 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 870/1753] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:06:47 2022] DELETE /identity/v3/projects/4f00d6f249f94245a6e012c3740ff646 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:48.299096 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac9df6-c3e2-4ab0-b361-77a140e17e72 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:48.299096 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-acac9df6-c3e2-4ab0-b361-77a140e17e72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:48.339304 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f55b0-6f93-4762-8962-c055e97c0c31 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:48.339973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f55b0-6f93-4762-8962-c055e97c0c31 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:48.340531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f55b0-6f93-4762-8962-c055e97c0c31 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:48.342576 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 871/1754] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 2 09:06:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:06:48.381777 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4611e52-7e4b-4523-ace0-c9b9309a46e2 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:48.383145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4611e52-7e4b-4523-ace0-c9b9309a46e2 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:48.383673 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4611e52-7e4b-4523-ace0-c9b9309a46e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:48.464494 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac9df6-c3e2-4ab0-b361-77a140e17e72 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:48.473414 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-f4611e52-7e4b-4523-ace0-c9b9309a46e2 None None] MFA Rules not processed for user `3191d804e8dc4b1a85f490319833ec07`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:48.484056 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f4611e52-7e4b-4523-ace0-c9b9309a46e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:48.597815 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:48.608158 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd1a82-0103-4703-9156-e42f5919cba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:48.609181 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd1a82-0103-4703-9156-e42f5919cba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:48.609613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd1a82-0103-4703-9156-e42f5919cba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:48.610069 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd1a82-0103-4703-9156-e42f5919cba2 admin admin] PATH_INFO: `/v3/users/cb983eb44d244da28614f9b3daaab2a6` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:48.611420 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd1a82-0103-4703-9156-e42f5919cba2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb983eb44d244da28614f9b3daaab2a6)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:48.613548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd1a82-0103-4703-9156-e42f5919cba2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:48.614052 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 872/1755] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 2 09:06:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:06:48.626625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4611e52-7e4b-4523-ace0-c9b9309a46e2 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:48.630049 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f4611e52-7e4b-4523-ace0-c9b9309a46e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:48.674134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4611e52-7e4b-4523-ace0-c9b9309a46e2 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:48.690272 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde5171-10e6-45ac-873b-49bf016bd6e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:48.691044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde5171-10e6-45ac-873b-49bf016bd6e4 service nova] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:48.691541 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde5171-10e6-45ac-873b-49bf016bd6e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:48.691832 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde5171-10e6-45ac-873b-49bf016bd6e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:48.704633 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde5171-10e6-45ac-873b-49bf016bd6e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:48.707245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde5171-10e6-45ac-873b-49bf016bd6e4 service nova] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:48.722500 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 873/1756] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:48.925330 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d7fd1a82-0103-4703-9156-e42f5919cba2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb983eb44d244da28614f9b3daaab2a6'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:48.935575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d7fd1a82-0103-4703-9156-e42f5919cba2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb983eb44d244da28614f9b3daaab2a6'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:48.938658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d7fd1a82-0103-4703-9156-e42f5919cba2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb983eb44d244da28614f9b3daaab2a6'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:48.964211 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d7fd1a82-0103-4703-9156-e42f5919cba2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb983eb44d244da28614f9b3daaab2a6'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:48.965235 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d7fd1a82-0103-4703-9156-e42f5919cba2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb983eb44d244da28614f9b3daaab2a6'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:49.103722 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 884/1757] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:06:48 2022] DELETE /identity/v3/users/cb983eb44d244da28614f9b3daaab2a6 => generated 0 bytes in 812 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:06:49.206069 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde5171-10e6-45ac-873b-49bf016bd6e4 service nova] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:49.207569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-2bde5171-10e6-45ac-873b-49bf016bd6e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:49.237334 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde5171-10e6-45ac-873b-49bf016bd6e4 service nova] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:49.319098 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2990a-4de4-4cd0-9e56-251b80543202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:49.320911 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2990a-4de4-4cd0-9e56-251b80543202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:49.321159 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2990a-4de4-4cd0-9e56-251b80543202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:49.321571 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2990a-4de4-4cd0-9e56-251b80543202 admin admin] PATH_INFO: `/v3/projects/b18d73a0533f43889ddc02e840309d4c` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:49.323649 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2990a-4de4-4cd0-9e56-251b80543202 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b18d73a0533f43889ddc02e840309d4c)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:49.326247 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2990a-4de4-4cd0-9e56-251b80543202 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:49.505123 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-0aa2990a-4de4-4cd0-9e56-251b80543202 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b18d73a0533f43889ddc02e840309d4c', 'request_id': 'req-0aa2990a-4de4-4cd0-9e56-251b80543202'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:49.508583 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:49.521137 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-0aa2990a-4de4-4cd0-9e56-251b80543202 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b18d73a0533f43889ddc02e840309d4c', 'request_id': 'req-0aa2990a-4de4-4cd0-9e56-251b80543202'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:49.532793 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-0aa2990a-4de4-4cd0-9e56-251b80543202 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b18d73a0533f43889ddc02e840309d4c', 'request_id': 'req-0aa2990a-4de4-4cd0-9e56-251b80543202'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:49.646283 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-0aa2990a-4de4-4cd0-9e56-251b80543202 admin admin] The token cache is being invalidate because project b18d73a0533f43889ddc02e840309d4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:06:49.646798 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-0aa2990a-4de4-4cd0-9e56-251b80543202 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:49.648001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-0aa2990a-4de4-4cd0-9e56-251b80543202 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:49.672477 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 874/1758] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:06:49 2022] DELETE /identity/v3/projects/b18d73a0533f43889ddc02e840309d4c => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:49.698513 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce585b-3315-493d-afca-f3ff3fc6ad7d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:49.698513 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce585b-3315-493d-afca-f3ff3fc6ad7d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:49.698513 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce585b-3315-493d-afca-f3ff3fc6ad7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:49.739133 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-35ce585b-3315-493d-afca-f3ff3fc6ad7d None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:49.743664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-35ce585b-3315-493d-afca-f3ff3fc6ad7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:49.858424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 885/1759] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:49.866412 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631e67e-0416-4915-89b0-0c453b15504c None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:49.866740 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631e67e-0416-4915-89b0-0c453b15504c None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:49.866963 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631e67e-0416-4915-89b0-0c453b15504c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:49.897042 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-0631e67e-0416-4915-89b0-0c453b15504c None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:49.898682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-0631e67e-0416-4915-89b0-0c453b15504c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:49.908881 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 875/1760] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:49.913764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce585b-3315-493d-afca-f3ff3fc6ad7d None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:49.934097 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce585b-3315-493d-afca-f3ff3fc6ad7d None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:49.938963 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b62128-50c6-40f4-88ff-97d309bda214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:49.963066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b62128-50c6-40f4-88ff-97d309bda214 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:49.963066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b62128-50c6-40f4-88ff-97d309bda214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:49.963066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b62128-50c6-40f4-88ff-97d309bda214 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:49.963066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b62128-50c6-40f4-88ff-97d309bda214 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:06:49.963066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b62128-50c6-40f4-88ff-97d309bda214 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:49.963066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b62128-50c6-40f4-88ff-97d309bda214 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:49.964161 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631e67e-0416-4915-89b0-0c453b15504c None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:49.967275 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-0631e67e-0416-4915-89b0-0c453b15504c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:49.977192 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 886/1761] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:06:49.983180 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b62128-50c6-40f4-88ff-97d309bda214 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:50.001416 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631e67e-0416-4915-89b0-0c453b15504c None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:50.005383 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b62128-50c6-40f4-88ff-97d309bda214 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:50.010295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd460674-2fb6-45e0-839b-2779aac3c7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:50.019462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd460674-2fb6-45e0-839b-2779aac3c7cc admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:50.019763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd460674-2fb6-45e0-839b-2779aac3c7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:50.020138 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd460674-2fb6-45e0-839b-2779aac3c7cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:50.022537 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb6ad36-863f-4725-9644-eef7776d8add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:50.022703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd460674-2fb6-45e0-839b-2779aac3c7cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:50.023279 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6ad36-863f-4725-9644-eef7776d8add admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:50.023643 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6ad36-863f-4725-9644-eef7776d8add admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:50.023996 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6ad36-863f-4725-9644-eef7776d8add admin admin] PATH_INFO: `/v3/projects/5989a0169d704e24a18d61271134656c` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:50.024754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd460674-2fb6-45e0-839b-2779aac3c7cc admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:50.025722 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb6ad36-863f-4725-9644-eef7776d8add admin admin] RBAC: Authorizing `identity:delete_project(project_id=5989a0169d704e24a18d61271134656c)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:50.031417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb6ad36-863f-4725-9644-eef7776d8add admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:50.173707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 887/1762] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:06:49 2022] POST /identity/v3/projects => generated 379 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:50.180404 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd460674-2fb6-45e0-839b-2779aac3c7cc admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:50.184738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9eb6ad36-863f-4725-9644-eef7776d8add admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5989a0169d704e24a18d61271134656c', 'request_id': 'req-9eb6ad36-863f-4725-9644-eef7776d8add'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:50.235484 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9eb6ad36-863f-4725-9644-eef7776d8add admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5989a0169d704e24a18d61271134656c', 'request_id': 'req-9eb6ad36-863f-4725-9644-eef7776d8add'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:50.262888 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9eb6ad36-863f-4725-9644-eef7776d8add admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5989a0169d704e24a18d61271134656c', 'request_id': 'req-9eb6ad36-863f-4725-9644-eef7776d8add'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:50.340777 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd460674-2fb6-45e0-839b-2779aac3c7cc admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:50.345680 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d980431-18ad-466d-a61a-87a77bee525b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:50.346341 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d980431-18ad-466d-a61a-87a77bee525b admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:50.346621 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d980431-18ad-466d-a61a-87a77bee525b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:50.346894 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d980431-18ad-466d-a61a-87a77bee525b admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:50.347601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d980431-18ad-466d-a61a-87a77bee525b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:50.349549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d980431-18ad-466d-a61a-87a77bee525b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:50.366442 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9eb6ad36-863f-4725-9644-eef7776d8add admin admin] The token cache is being invalidate because project 5989a0169d704e24a18d61271134656c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:06:50.366834 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9eb6ad36-863f-4725-9644-eef7776d8add admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:50.367754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9eb6ad36-863f-4725-9644-eef7776d8add admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:50.369466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 876/1763] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:06:49 2022] DELETE /identity/v3/projects/5989a0169d704e24a18d61271134656c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:06:50.377812 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b93fb-9189-4024-958a-a967ea1ae42f None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:50.378460 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b93fb-9189-4024-958a-a967ea1ae42f None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:50.378947 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b93fb-9189-4024-958a-a967ea1ae42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:50.400187 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-1c0b93fb-9189-4024-958a-a967ea1ae42f None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:50.401914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-1c0b93fb-9189-4024-958a-a967ea1ae42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:50.410528 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 877/1764] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:50.418651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5e5b7-62ba-463b-8f36-d287efd58826 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:50.419151 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5e5b7-62ba-463b-8f36-d287efd58826 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:50.419265 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5e5b7-62ba-463b-8f36-d287efd58826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:50.443886 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-11c5e5b7-62ba-463b-8f36-d287efd58826 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:50.445405 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-11c5e5b7-62ba-463b-8f36-d287efd58826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:50.458415 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 888/1765] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:06:50 2022] POST /identity/v3/users => generated 290 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:50.464766 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d980431-18ad-466d-a61a-87a77bee525b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:50.464766 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9d980431-18ad-466d-a61a-87a77bee525b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:50.473496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 878/1766] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:06:50.478097 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5e5b7-62ba-463b-8f36-d287efd58826 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:50.481323 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d980431-18ad-466d-a61a-87a77bee525b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:50.489706 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb3fcb6-b3f2-4379-baea-7aaeccabc669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:50.490339 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3fcb6-b3f2-4379-baea-7aaeccabc669 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:50.490633 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3fcb6-b3f2-4379-baea-7aaeccabc669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:50.490921 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3fcb6-b3f2-4379-baea-7aaeccabc669 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:50.491572 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb3fcb6-b3f2-4379-baea-7aaeccabc669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:50.493200 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb3fcb6-b3f2-4379-baea-7aaeccabc669 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:50.501768 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5e5b7-62ba-463b-8f36-d287efd58826 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:50.515122 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 889/1767] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:50.519754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293e3e9-0358-41a0-94b2-d1c590e91903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:50.520293 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb3fcb6-b3f2-4379-baea-7aaeccabc669 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:50.520570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293e3e9-0358-41a0-94b2-d1c590e91903 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:50.520883 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293e3e9-0358-41a0-94b2-d1c590e91903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:50.521156 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293e3e9-0358-41a0-94b2-d1c590e91903 admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:50.523885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293e3e9-0358-41a0-94b2-d1c590e91903 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:06:50.524177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293e3e9-0358-41a0-94b2-d1c590e91903 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:50.529157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3293e3e9-0358-41a0-94b2-d1c590e91903 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:50.542457 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb3fcb6-b3f2-4379-baea-7aaeccabc669 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:50.547335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496d8f5-a727-4c21-9845-9cb245a2ffd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:50.548237 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496d8f5-a727-4c21-9845-9cb245a2ffd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:50.548663 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496d8f5-a727-4c21-9845-9cb245a2ffd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:50.549017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496d8f5-a727-4c21-9845-9cb245a2ffd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:50.549797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496d8f5-a727-4c21-9845-9cb245a2ffd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:50.551855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496d8f5-a727-4c21-9845-9cb245a2ffd7 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:50.559125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 879/1768] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:06:50.567148 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293e3e9-0358-41a0-94b2-d1c590e91903 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:50.577640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 890/1769] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:50.582338 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496d8f5-a727-4c21-9845-9cb245a2ffd7 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:50.594292 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496d8f5-a727-4c21-9845-9cb245a2ffd7 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:50.598117 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293e3e9-0358-41a0-94b2-d1c590e91903 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:50.610579 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d8968-c74b-4985-9023-2aee334fb378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:50.611274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d8968-c74b-4985-9023-2aee334fb378 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:50.611578 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d8968-c74b-4985-9023-2aee334fb378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:50.611846 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d8968-c74b-4985-9023-2aee334fb378 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:50.614015 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d8968-c74b-4985-9023-2aee334fb378 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:50.614613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc9a4e-fe6f-4bcb-ae43-0e0af902bf0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:50.616187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc9a4e-fe6f-4bcb-ae43-0e0af902bf0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:50.616986 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d8968-c74b-4985-9023-2aee334fb378 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:50.618497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc9a4e-fe6f-4bcb-ae43-0e0af902bf0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:50.619296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc9a4e-fe6f-4bcb-ae43-0e0af902bf0a admin admin] PATH_INFO: `/v3/projects/6956616275cd4ebfa23f551e04cc4710/users/05989a39c40b4a2d820c73ad17edd2e9/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:50.647433 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc9a4e-fe6f-4bcb-ae43-0e0af902bf0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6956616275cd4ebfa23f551e04cc4710, user_id=05989a39c40b4a2d820c73ad17edd2e9, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:50.650604 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc9a4e-fe6f-4bcb-ae43-0e0af902bf0a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:50.721446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 880/1770] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:06:50 2022] POST /identity/v3/projects => generated 367 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:06:50.724042 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d8968-c74b-4985-9023-2aee334fb378 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:50.760255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 891/1771] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:50 2022] PUT /identity/v3/projects/6956616275cd4ebfa23f551e04cc4710/users/05989a39c40b4a2d820c73ad17edd2e9/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:50.763481 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1819c-15e7-4218-aa87-9b7febcb84d2 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:50.764112 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1819c-15e7-4218-aa87-9b7febcb84d2 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:50.764396 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1819c-15e7-4218-aa87-9b7febcb84d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:50.801979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-38b1819c-15e7-4218-aa87-9b7febcb84d2 None None] MFA Rules not processed for user `05989a39c40b4a2d820c73ad17edd2e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:50.803369 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-38b1819c-15e7-4218-aa87-9b7febcb84d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:50.913004 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d8968-c74b-4985-9023-2aee334fb378 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:50.934437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 892/1772] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:50.939388 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1819c-15e7-4218-aa87-9b7febcb84d2 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:51.008049 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64c540-24d5-4178-8669-f6d3ee288101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:51.008991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64c540-24d5-4178-8669-f6d3ee288101 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:51.009427 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64c540-24d5-4178-8669-f6d3ee288101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:51.010014 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64c540-24d5-4178-8669-f6d3ee288101 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:51.011612 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64c540-24d5-4178-8669-f6d3ee288101 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:51.014148 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64c540-24d5-4178-8669-f6d3ee288101 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:51.029979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1819c-15e7-4218-aa87-9b7febcb84d2 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:51.035153 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4c8cf-2978-4187-ab14-fc0da26fbdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:51.035784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4c8cf-2978-4187-ab14-fc0da26fbdb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:51.036225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4c8cf-2978-4187-ab14-fc0da26fbdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:51.036505 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4c8cf-2978-4187-ab14-fc0da26fbdb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:51.037358 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4c8cf-2978-4187-ab14-fc0da26fbdb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:51.039398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4c8cf-2978-4187-ab14-fc0da26fbdb4 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:51.101283 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 881/1773] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:06:50 2022] POST /identity/v3/users => generated 284 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:06:51.106040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64c540-24d5-4178-8669-f6d3ee288101 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:51.116371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64c540-24d5-4178-8669-f6d3ee288101 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:51.122039 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033ffea-b38e-4721-a0e3-0468ceb53264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:51.122678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033ffea-b38e-4721-a0e3-0468ceb53264 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:51.122958 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033ffea-b38e-4721-a0e3-0468ceb53264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:51.123234 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8033ffea-b38e-4721-a0e3-0468ceb53264 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:51.123933 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8033ffea-b38e-4721-a0e3-0468ceb53264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:51.125546 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8033ffea-b38e-4721-a0e3-0468ceb53264 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:51.139385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 882/1774] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:06:51.140967 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033ffea-b38e-4721-a0e3-0468ceb53264 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:51.147026 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 893/1775] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:06:50 2022] POST /identity/v3/projects => generated 379 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:06:51.151067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4c8cf-2978-4187-ab14-fc0da26fbdb4 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:51.152549 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8033ffea-b38e-4721-a0e3-0468ceb53264 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:51.222871 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4c8cf-2978-4187-ab14-fc0da26fbdb4 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:51.225758 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f1994-c241-4fc9-b1b8-5ff27651ee70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:51.226483 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f1994-c241-4fc9-b1b8-5ff27651ee70 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:51.226766 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f1994-c241-4fc9-b1b8-5ff27651ee70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:51.227043 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1f1994-c241-4fc9-b1b8-5ff27651ee70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:51.230261 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f1994-c241-4fc9-b1b8-5ff27651ee70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:51.235640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf29327-8a4b-453b-a355-e5480bcbd377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:51.235640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf29327-8a4b-453b-a355-e5480bcbd377 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:51.235640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf29327-8a4b-453b-a355-e5480bcbd377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:51.235640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf29327-8a4b-453b-a355-e5480bcbd377 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:51.236026 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1f1994-c241-4fc9-b1b8-5ff27651ee70 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:51.236495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf29327-8a4b-453b-a355-e5480bcbd377 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:51.236495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf29327-8a4b-453b-a355-e5480bcbd377 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:51.243694 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 883/1776] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:51.248301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f1994-c241-4fc9-b1b8-5ff27651ee70 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:51.259136 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1f1994-c241-4fc9-b1b8-5ff27651ee70 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:51.264728 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b43640-bd81-44a9-8418-ca489ca2f23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:51.265425 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b43640-bd81-44a9-8418-ca489ca2f23f admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:51.265711 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b43640-bd81-44a9-8418-ca489ca2f23f admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:51.266022 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b43640-bd81-44a9-8418-ca489ca2f23f admin admin] PATH_INFO: `/v3/projects/2faa2bc3068849959e87189502f18f07/users/3d78578a400649b6a866444e4ccb913a/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:51.282967 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b43640-bd81-44a9-8418-ca489ca2f23f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2faa2bc3068849959e87189502f18f07, user_id=3d78578a400649b6a866444e4ccb913a, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:51.285651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b43640-bd81-44a9-8418-ca489ca2f23f admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:51.310851 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 894/1777] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:06:51 2022] POST /identity/v3/users => generated 290 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:06:51.311981 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf29327-8a4b-453b-a355-e5480bcbd377 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:51.321805 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf29327-8a4b-453b-a355-e5480bcbd377 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:51.327713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ddf1f-172b-42c7-8aab-ed2df91a21a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:51.328370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ddf1f-172b-42c7-8aab-ed2df91a21a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:51.328650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ddf1f-172b-42c7-8aab-ed2df91a21a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:51.328924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ddf1f-172b-42c7-8aab-ed2df91a21a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:51.329875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ddf1f-172b-42c7-8aab-ed2df91a21a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:51.331621 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ddf1f-172b-42c7-8aab-ed2df91a21a3 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:51.346730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 895/1778] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:51.347931 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ddf1f-172b-42c7-8aab-ed2df91a21a3 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:51.358132 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ddf1f-172b-42c7-8aab-ed2df91a21a3 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:51.362994 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a6cce-e763-4cdc-87bc-b3874f00ece7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:51.363656 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a6cce-e763-4cdc-87bc-b3874f00ece7 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:51.363939 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a6cce-e763-4cdc-87bc-b3874f00ece7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:51.364216 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a6cce-e763-4cdc-87bc-b3874f00ece7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:51.364947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a6cce-e763-4cdc-87bc-b3874f00ece7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:51.366585 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5a6cce-e763-4cdc-87bc-b3874f00ece7 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:51.378477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 896/1779] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:06:51.382493 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a6cce-e763-4cdc-87bc-b3874f00ece7 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:51.393397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a6cce-e763-4cdc-87bc-b3874f00ece7 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:51.393993 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 884/1780] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:51 2022] PUT /identity/v3/projects/2faa2bc3068849959e87189502f18f07/users/3d78578a400649b6a866444e4ccb913a/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:06:51.394981 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b43640-bd81-44a9-8418-ca489ca2f23f admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:51.396519 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e5b43640-bd81-44a9-8418-ca489ca2f23f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:51.456882 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf1682-5041-45f1-9030-201efba54a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:51.457581 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf1682-5041-45f1-9030-201efba54a4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:51.457866 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf1682-5041-45f1-9030-201efba54a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:51.458173 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf1682-5041-45f1-9030-201efba54a4e admin admin] PATH_INFO: `/v3/projects/85c2adc914454a7f86b6ec24754a822e/users/81e41fe5ac9b4b2abc54b21942a934d9/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:51.469039 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b43640-bd81-44a9-8418-ca489ca2f23f admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:51.473295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf1682-5041-45f1-9030-201efba54a4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=85c2adc914454a7f86b6ec24754a822e, user_id=81e41fe5ac9b4b2abc54b21942a934d9, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:51.473798 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ca323-80ab-44f9-87ea-d3677d623584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:51.474629 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ca323-80ab-44f9-87ea-d3677d623584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:51.474910 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ca323-80ab-44f9-87ea-d3677d623584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:51.479097 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ca323-80ab-44f9-87ea-d3677d623584 admin admin] PATH_INFO: `/v3/users/95347780190343d391fdc7752d171c8d` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:51.479203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf1682-5041-45f1-9030-201efba54a4e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:51.479305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ca323-80ab-44f9-87ea-d3677d623584 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95347780190343d391fdc7752d171c8d)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:51.479305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ca323-80ab-44f9-87ea-d3677d623584 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:51.547551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 897/1781] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:51 2022] PUT /identity/v3/projects/85c2adc914454a7f86b6ec24754a822e/users/81e41fe5ac9b4b2abc54b21942a934d9/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:06:51.549196 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c34d4b-c936-422f-8f92-65457926a90b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:51.549490 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c34d4b-c936-422f-8f92-65457926a90b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:51.549772 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c34d4b-c936-422f-8f92-65457926a90b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:51.580039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-22c34d4b-c936-422f-8f92-65457926a90b None None] MFA Rules not processed for user `3d78578a400649b6a866444e4ccb913a`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:51.581260 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-22c34d4b-c936-422f-8f92-65457926a90b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:51.645922 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 898/1782] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:51.647361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162a268-9d47-44a9-89d2-234eb066d15f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:51.647654 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162a268-9d47-44a9-89d2-234eb066d15f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:51.647932 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8162a268-9d47-44a9-89d2-234eb066d15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:51.675097 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-8162a268-9d47-44a9-89d2-234eb066d15f None None] MFA Rules not processed for user `81e41fe5ac9b4b2abc54b21942a934d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:51.676499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8162a268-9d47-44a9-89d2-234eb066d15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:51.726633 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-3d9ca323-80ab-44f9-87ea-d3677d623584 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95347780190343d391fdc7752d171c8d'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:51.734296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-3d9ca323-80ab-44f9-87ea-d3677d623584 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95347780190343d391fdc7752d171c8d'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:51.734970 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-3d9ca323-80ab-44f9-87ea-d3677d623584 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95347780190343d391fdc7752d171c8d'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:51.739487 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 899/1783] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:51.744533 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162a268-9d47-44a9-89d2-234eb066d15f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:51.811760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8162a268-9d47-44a9-89d2-234eb066d15f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:51.816435 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867696c-1724-44af-a459-210ef49b4054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:51.817057 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867696c-1724-44af-a459-210ef49b4054 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:51.817337 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867696c-1724-44af-a459-210ef49b4054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:51.817611 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867696c-1724-44af-a459-210ef49b4054 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:51.818311 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867696c-1724-44af-a459-210ef49b4054 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:51.820320 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867696c-1724-44af-a459-210ef49b4054 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:51.873094 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-3d9ca323-80ab-44f9-87ea-d3677d623584 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95347780190343d391fdc7752d171c8d'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:51.894445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-3d9ca323-80ab-44f9-87ea-d3677d623584 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95347780190343d391fdc7752d171c8d'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:51.902395 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 885/1784] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:06:51 2022] DELETE /identity/v3/users/95347780190343d391fdc7752d171c8d => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:51.903795 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ca323-80ab-44f9-87ea-d3677d623584 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:51.905403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-3d9ca323-80ab-44f9-87ea-d3677d623584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:51.971847 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 900/1785] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:06:51 2022] POST /identity/v3/projects => generated 369 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:51.971847 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1d61e-6559-4221-95dc-80ec1adb8861 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:51.971847 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1d61e-6559-4221-95dc-80ec1adb8861 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:51.971847 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1d61e-6559-4221-95dc-80ec1adb8861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:51.998796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-08b1d61e-6559-4221-95dc-80ec1adb8861 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:52.000556 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-08b1d61e-6559-4221-95dc-80ec1adb8861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:52.023936 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ca323-80ab-44f9-87ea-d3677d623584 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:52.082924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 901/1786] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:52.082924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1d61e-6559-4221-95dc-80ec1adb8861 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:52.082924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-08b1d61e-6559-4221-95dc-80ec1adb8861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:52.101350 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1d61e-6559-4221-95dc-80ec1adb8861 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:52.107573 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ef9b8-5562-4445-8f2f-8fcb2c3b7ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:52.108595 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ef9b8-5562-4445-8f2f-8fcb2c3b7ae1 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:52.108595 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ef9b8-5562-4445-8f2f-8fcb2c3b7ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:52.108939 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ef9b8-5562-4445-8f2f-8fcb2c3b7ae1 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:52.112837 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ef9b8-5562-4445-8f2f-8fcb2c3b7ae1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:52.112837 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261ef9b8-5562-4445-8f2f-8fcb2c3b7ae1 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:52.119589 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcdc7fe-fd78-4e4b-ba5b-59c2702d3143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:52.120461 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdc7fe-fd78-4e4b-ba5b-59c2702d3143 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:52.120922 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdc7fe-fd78-4e4b-ba5b-59c2702d3143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:52.121353 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcdc7fe-fd78-4e4b-ba5b-59c2702d3143 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:52.122237 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcdc7fe-fd78-4e4b-ba5b-59c2702d3143 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:52.124528 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcdc7fe-fd78-4e4b-ba5b-59c2702d3143 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:52.183086 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 902/1787] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:06:52 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:52.186866 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ef9b8-5562-4445-8f2f-8fcb2c3b7ae1 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:52.186866 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-261ef9b8-5562-4445-8f2f-8fcb2c3b7ae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:52.217016 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 886/1788] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:06:51 2022] POST /identity/v3/projects => generated 377 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:52.218343 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcdc7fe-fd78-4e4b-ba5b-59c2702d3143 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:52.254866 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261ef9b8-5562-4445-8f2f-8fcb2c3b7ae1 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:52.290788 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcdc7fe-fd78-4e4b-ba5b-59c2702d3143 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:52.298939 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ff42e-044d-4924-9c44-69bd2e8283dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:52.301674 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ff42e-044d-4924-9c44-69bd2e8283dc admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:52.301674 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ff42e-044d-4924-9c44-69bd2e8283dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:52.301674 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ff42e-044d-4924-9c44-69bd2e8283dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:52.301674 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862ff42e-044d-4924-9c44-69bd2e8283dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:52.303589 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862ff42e-044d-4924-9c44-69bd2e8283dc admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:52.315187 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 887/1789] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:52.316180 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ff42e-044d-4924-9c44-69bd2e8283dc admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:52.319763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53dd8d8-99b4-4eb8-a3e3-b8dde0179d15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:52.320478 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53dd8d8-99b4-4eb8-a3e3-b8dde0179d15 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:52.320859 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53dd8d8-99b4-4eb8-a3e3-b8dde0179d15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:52.321219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53dd8d8-99b4-4eb8-a3e3-b8dde0179d15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:52.329218 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ff42e-044d-4924-9c44-69bd2e8283dc admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:52.332659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53dd8d8-99b4-4eb8-a3e3-b8dde0179d15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:52.334239 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53dd8d8-99b4-4eb8-a3e3-b8dde0179d15 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:52.336341 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dda488-c1da-4a41-a7b4-68b4d326dca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:52.336341 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dda488-c1da-4a41-a7b4-68b4d326dca4 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:52.336341 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dda488-c1da-4a41-a7b4-68b4d326dca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:52.336341 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dda488-c1da-4a41-a7b4-68b4d326dca4 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:52.336741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dda488-c1da-4a41-a7b4-68b4d326dca4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:52.338743 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dda488-c1da-4a41-a7b4-68b4d326dca4 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:52.345050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 903/1790] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:52.345713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53dd8d8-99b4-4eb8-a3e3-b8dde0179d15 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:52.355231 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53dd8d8-99b4-4eb8-a3e3-b8dde0179d15 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:52.360999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca82ea-5c37-491b-94aa-217538209259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:52.365651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca82ea-5c37-491b-94aa-217538209259 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:52.365651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca82ea-5c37-491b-94aa-217538209259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:52.365651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca82ea-5c37-491b-94aa-217538209259 admin admin] PATH_INFO: `/v3/projects/2469c01d9a7e4c93ad62310367eff906/users/da3a0adb3761484bb1e7a852461a448a/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:52.397184 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faca82ea-5c37-491b-94aa-217538209259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2469c01d9a7e4c93ad62310367eff906, user_id=da3a0adb3761484bb1e7a852461a448a, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:52.400603 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faca82ea-5c37-491b-94aa-217538209259 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:52.415026 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:52.460642 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 888/1791] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:06:52 2022] POST /identity/v3/users => generated 295 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:52.464681 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dda488-c1da-4a41-a7b4-68b4d326dca4 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:52.474596 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dda488-c1da-4a41-a7b4-68b4d326dca4 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:52.479701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165bb674-6f43-4609-9760-595903464f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:52.480322 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bb674-6f43-4609-9760-595903464f24 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:52.480601 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bb674-6f43-4609-9760-595903464f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:52.480876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bb674-6f43-4609-9760-595903464f24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:52.481564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165bb674-6f43-4609-9760-595903464f24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:52.483241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165bb674-6f43-4609-9760-595903464f24 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:52.490508 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 904/1792] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:52 2022] PUT /identity/v3/projects/2469c01d9a7e4c93ad62310367eff906/users/da3a0adb3761484bb1e7a852461a448a/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:52.499058 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 889/1793] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:06:52.499233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca82ea-5c37-491b-94aa-217538209259 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:52.500105 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165bb674-6f43-4609-9760-595903464f24 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:52.511022 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165bb674-6f43-4609-9760-595903464f24 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:52.564060 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca82ea-5c37-491b-94aa-217538209259 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:52.568864 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d56ad6-f0c5-470c-a880-e95a9a16370e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:52.569076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348175ea-9cb9-41db-8bea-13b84dfa432a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:52.569465 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d56ad6-f0c5-470c-a880-e95a9a16370e admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:52.569596 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348175ea-9cb9-41db-8bea-13b84dfa432a admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:52.569743 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d56ad6-f0c5-470c-a880-e95a9a16370e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:52.569831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348175ea-9cb9-41db-8bea-13b84dfa432a admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:52.570090 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d56ad6-f0c5-470c-a880-e95a9a16370e admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:52.570213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348175ea-9cb9-41db-8bea-13b84dfa432a admin admin] PATH_INFO: `/v3/projects/5c527d333b4148e39b11bedb2e1602c2/users/3d033cd703c342809e214aef8fa6aa72/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:52.570790 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d56ad6-f0c5-470c-a880-e95a9a16370e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:52.572408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d56ad6-f0c5-470c-a880-e95a9a16370e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:52.589789 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 905/1794] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:06:52.595774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348175ea-9cb9-41db-8bea-13b84dfa432a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c527d333b4148e39b11bedb2e1602c2, user_id=3d033cd703c342809e214aef8fa6aa72, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:52.596502 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d56ad6-f0c5-470c-a880-e95a9a16370e admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:52.599935 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348175ea-9cb9-41db-8bea-13b84dfa432a admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:52.608734 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d56ad6-f0c5-470c-a880-e95a9a16370e admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:52.613738 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68830121-fb77-4554-bd42-82d87c7aca68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:52.614574 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68830121-fb77-4554-bd42-82d87c7aca68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:52.615077 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68830121-fb77-4554-bd42-82d87c7aca68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:52.615192 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68830121-fb77-4554-bd42-82d87c7aca68 admin admin] PATH_INFO: `/v3/projects/2469c01d9a7e4c93ad62310367eff906/users/da3a0adb3761484bb1e7a852461a448a/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:52.617251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68830121-fb77-4554-bd42-82d87c7aca68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2469c01d9a7e4c93ad62310367eff906, user_id=da3a0adb3761484bb1e7a852461a448a, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:52.622245 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68830121-fb77-4554-bd42-82d87c7aca68 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:52.635089 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 906/1795] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:52 2022] PUT /identity/v3/projects/2469c01d9a7e4c93ad62310367eff906/users/da3a0adb3761484bb1e7a852461a448a/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:52.652900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecfc1d-3e5a-4ee7-9505-a63496411b6c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:52.653216 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecfc1d-3e5a-4ee7-9505-a63496411b6c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:52.653493 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecfc1d-3e5a-4ee7-9505-a63496411b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:52.683065 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-37ecfc1d-3e5a-4ee7-9505-a63496411b6c None None] MFA Rules not processed for user `da3a0adb3761484bb1e7a852461a448a`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:52.684282 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-37ecfc1d-3e5a-4ee7-9505-a63496411b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:52.701920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 890/1796] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:52 2022] PUT /identity/v3/projects/5c527d333b4148e39b11bedb2e1602c2/users/3d033cd703c342809e214aef8fa6aa72/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:06:52.705578 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348175ea-9cb9-41db-8bea-13b84dfa432a admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:52.761771 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 907/1797] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:52 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:52.773194 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecfc1d-3e5a-4ee7-9505-a63496411b6c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:52.774683 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-37ecfc1d-3e5a-4ee7-9505-a63496411b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:52.782696 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348175ea-9cb9-41db-8bea-13b84dfa432a admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:52.787928 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f5fdc-19a5-4838-b21b-a9faa66780dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:52.788700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f5fdc-19a5-4838-b21b-a9faa66780dd admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:52.789046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f5fdc-19a5-4838-b21b-a9faa66780dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:52.789989 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f5fdc-19a5-4838-b21b-a9faa66780dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:52.790188 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f5fdc-19a5-4838-b21b-a9faa66780dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:52.792461 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f5fdc-19a5-4838-b21b-a9faa66780dd admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:52.803667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 891/1798] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:06:52.808354 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f5fdc-19a5-4838-b21b-a9faa66780dd admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:52.819274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f5fdc-19a5-4838-b21b-a9faa66780dd admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:52.824877 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b5ca41-14d1-4c14-9803-bca8352d8ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:52.828024 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5ca41-14d1-4c14-9803-bca8352d8ed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:52.828024 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5ca41-14d1-4c14-9803-bca8352d8ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:52.828024 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b5ca41-14d1-4c14-9803-bca8352d8ed8 admin admin] PATH_INFO: `/v3/projects/5c527d333b4148e39b11bedb2e1602c2/users/3d033cd703c342809e214aef8fa6aa72/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:52.828262 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5ca41-14d1-4c14-9803-bca8352d8ed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c527d333b4148e39b11bedb2e1602c2, user_id=3d033cd703c342809e214aef8fa6aa72, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:52.831066 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b5ca41-14d1-4c14-9803-bca8352d8ed8 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:52.838295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecfc1d-3e5a-4ee7-9505-a63496411b6c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:52.841876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 892/1799] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:52 2022] PUT /identity/v3/projects/5c527d333b4148e39b11bedb2e1602c2/users/3d033cd703c342809e214aef8fa6aa72/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:06:52.853416 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668f2ed-9bf4-40f0-a3b1-11517e0b80a5 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:52.853774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668f2ed-9bf4-40f0-a3b1-11517e0b80a5 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:52.854265 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5668f2ed-9bf4-40f0-a3b1-11517e0b80a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:52.891005 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-5668f2ed-9bf4-40f0-a3b1-11517e0b80a5 None None] MFA Rules not processed for user `3d033cd703c342809e214aef8fa6aa72`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:52.892310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-5668f2ed-9bf4-40f0-a3b1-11517e0b80a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:52.896297 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ff415-ec93-4cf3-be34-761b5d922f38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:52.896930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ff415-ec93-4cf3-be34-761b5d922f38 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:52.897213 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ff415-ec93-4cf3-be34-761b5d922f38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:52.897493 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ff415-ec93-4cf3-be34-761b5d922f38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:52.960888 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38ff415-ec93-4cf3-be34-761b5d922f38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:52.962440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38ff415-ec93-4cf3-be34-761b5d922f38 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:52.965778 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 893/1800] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:06:52.972960 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 908/1801] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:06:52.975884 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668f2ed-9bf4-40f0-a3b1-11517e0b80a5 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:52.977335 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-5668f2ed-9bf4-40f0-a3b1-11517e0b80a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:52.977579 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ff415-ec93-4cf3-be34-761b5d922f38 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:52.980037 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a38ff415-ec93-4cf3-be34-761b5d922f38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:52.990558 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ff415-ec93-4cf3-be34-761b5d922f38 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:52.995929 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d36cb1-8ece-42fd-b568-ea9b2d667b33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:52.996486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d36cb1-8ece-42fd-b568-ea9b2d667b33 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:52.996769 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d36cb1-8ece-42fd-b568-ea9b2d667b33 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:52.997117 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d36cb1-8ece-42fd-b568-ea9b2d667b33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:53.019213 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d36cb1-8ece-42fd-b568-ea9b2d667b33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:53.024082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d36cb1-8ece-42fd-b568-ea9b2d667b33 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:53.037437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 909/1802] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:06:53.084611 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d36cb1-8ece-42fd-b568-ea9b2d667b33 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:53.096774 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d36cb1-8ece-42fd-b568-ea9b2d667b33 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:53.102128 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcec6be-655f-444f-b49a-a36f929b0e1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:53.103055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcec6be-655f-444f-b49a-a36f929b0e1c service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:53.105159 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcec6be-655f-444f-b49a-a36f929b0e1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:53.105971 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcec6be-655f-444f-b49a-a36f929b0e1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:53.108792 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6bcec6be-655f-444f-b49a-a36f929b0e1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:53.148157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5668f2ed-9bf4-40f0-a3b1-11517e0b80a5 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:53.168518 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c29465-ab29-4b04-b3f1-c07c8655a6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:53.169570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c29465-ab29-4b04-b3f1-c07c8655a6da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:53.170002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c29465-ab29-4b04-b3f1-c07c8655a6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:53.170287 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c29465-ab29-4b04-b3f1-c07c8655a6da admin admin] PATH_INFO: `/v3/projects/bef90b0b9ccb444b92430cc079fc846a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:53.173070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c29465-ab29-4b04-b3f1-c07c8655a6da admin admin] RBAC: Authorizing `identity:delete_project(project_id=bef90b0b9ccb444b92430cc079fc846a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:53.179082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c29465-ab29-4b04-b3f1-c07c8655a6da admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:53.241959 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcec6be-655f-444f-b49a-a36f929b0e1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:53.243560 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:53.245105 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcec6be-655f-444f-b49a-a36f929b0e1c service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:53.265179 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 910/1803] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:06:53.266601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcec6be-655f-444f-b49a-a36f929b0e1c service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:53.280876 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcec6be-655f-444f-b49a-a36f929b0e1c service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:53.286236 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418d7f4-1f0b-4dcb-a8bb-39d21dad76f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:53.287146 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418d7f4-1f0b-4dcb-a8bb-39d21dad76f1 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:53.287574 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418d7f4-1f0b-4dcb-a8bb-39d21dad76f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:53.288092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418d7f4-1f0b-4dcb-a8bb-39d21dad76f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:53.290731 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2418d7f4-1f0b-4dcb-a8bb-39d21dad76f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:53.454820 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e1c29465-ab29-4b04-b3f1-c07c8655a6da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bef90b0b9ccb444b92430cc079fc846a', 'request_id': 'req-e1c29465-ab29-4b04-b3f1-c07c8655a6da'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:53.470654 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e1c29465-ab29-4b04-b3f1-c07c8655a6da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bef90b0b9ccb444b92430cc079fc846a', 'request_id': 'req-e1c29465-ab29-4b04-b3f1-c07c8655a6da'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:53.488040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e1c29465-ab29-4b04-b3f1-c07c8655a6da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bef90b0b9ccb444b92430cc079fc846a', 'request_id': 'req-e1c29465-ab29-4b04-b3f1-c07c8655a6da'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:53.496104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418d7f4-1f0b-4dcb-a8bb-39d21dad76f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:53.503233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2418d7f4-1f0b-4dcb-a8bb-39d21dad76f1 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:53.539139 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 911/1804] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:06:53.577829 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e1c29465-ab29-4b04-b3f1-c07c8655a6da admin admin] The token cache is being invalidate because project bef90b0b9ccb444b92430cc079fc846a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:06:53.578278 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e1c29465-ab29-4b04-b3f1-c07c8655a6da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:53.579654 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e1c29465-ab29-4b04-b3f1-c07c8655a6da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:53.586634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 894/1805] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:06:52 2022] DELETE /identity/v3/projects/bef90b0b9ccb444b92430cc079fc846a => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:53.590937 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418d7f4-1f0b-4dcb-a8bb-39d21dad76f1 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:53.622758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418d7f4-1f0b-4dcb-a8bb-39d21dad76f1 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:53.759217 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42276f-3801-430b-90c8-9e53b6640e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:53.760717 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42276f-3801-430b-90c8-9e53b6640e3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:53.762005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42276f-3801-430b-90c8-9e53b6640e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:53.762402 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a42276f-3801-430b-90c8-9e53b6640e3a admin admin] PATH_INFO: `/v3/users/12dcab0775e048a09ce82a37afb2daea` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:53.764696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a42276f-3801-430b-90c8-9e53b6640e3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=12dcab0775e048a09ce82a37afb2daea)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:53.766864 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a42276f-3801-430b-90c8-9e53b6640e3a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:53.970401 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6a42276f-3801-430b-90c8-9e53b6640e3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12dcab0775e048a09ce82a37afb2daea'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:53.980294 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6a42276f-3801-430b-90c8-9e53b6640e3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12dcab0775e048a09ce82a37afb2daea'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:53.980920 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6a42276f-3801-430b-90c8-9e53b6640e3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12dcab0775e048a09ce82a37afb2daea'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:53.995958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6a42276f-3801-430b-90c8-9e53b6640e3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12dcab0775e048a09ce82a37afb2daea'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:54.003242 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6a42276f-3801-430b-90c8-9e53b6640e3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12dcab0775e048a09ce82a37afb2daea'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:54.125285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 912/1806] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:06:53 2022] DELETE /identity/v3/users/12dcab0775e048a09ce82a37afb2daea => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:55.271933 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c29465-ab29-4b04-b3f1-c07c8655a6da admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:55.276565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e1c29465-ab29-4b04-b3f1-c07c8655a6da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:55.293381 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c29465-ab29-4b04-b3f1-c07c8655a6da admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:55.403178 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c2f9f-cd14-41fd-9033-4978242a54bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:55.409299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c2f9f-cd14-41fd-9033-4978242a54bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:55.409299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c2f9f-cd14-41fd-9033-4978242a54bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:55.409299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c2f9f-cd14-41fd-9033-4978242a54bd admin admin] PATH_INFO: `/v3/projects/4d3ac70cda5846aa8cd5b292dae06149` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:55.409299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c2f9f-cd14-41fd-9033-4978242a54bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d3ac70cda5846aa8cd5b292dae06149)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:55.409299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c2f9f-cd14-41fd-9033-4978242a54bd admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:55.502267 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:55.574897 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-339c2f9f-cd14-41fd-9033-4978242a54bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d3ac70cda5846aa8cd5b292dae06149', 'request_id': 'req-339c2f9f-cd14-41fd-9033-4978242a54bd'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:55.587933 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-339c2f9f-cd14-41fd-9033-4978242a54bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d3ac70cda5846aa8cd5b292dae06149', 'request_id': 'req-339c2f9f-cd14-41fd-9033-4978242a54bd'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:55.613108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-339c2f9f-cd14-41fd-9033-4978242a54bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d3ac70cda5846aa8cd5b292dae06149', 'request_id': 'req-339c2f9f-cd14-41fd-9033-4978242a54bd'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:55.708148 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-339c2f9f-cd14-41fd-9033-4978242a54bd admin admin] The token cache is being invalidate because project 4d3ac70cda5846aa8cd5b292dae06149 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:06:55.708442 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-339c2f9f-cd14-41fd-9033-4978242a54bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:55.709650 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-339c2f9f-cd14-41fd-9033-4978242a54bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:55.718324 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42276f-3801-430b-90c8-9e53b6640e3a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:55.718884 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 895/1807] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:06:55 2022] DELETE /identity/v3/projects/4d3ac70cda5846aa8cd5b292dae06149 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:55.720690 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6a42276f-3801-430b-90c8-9e53b6640e3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:55.733022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a42276f-3801-430b-90c8-9e53b6640e3a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:55.824525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03977f-5bc4-44b5-8ea6-94a2c162a84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:55.825379 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03977f-5bc4-44b5-8ea6-94a2c162a84f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:55.825762 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03977f-5bc4-44b5-8ea6-94a2c162a84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:55.828085 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03977f-5bc4-44b5-8ea6-94a2c162a84f admin admin] PATH_INFO: `/v3/users/60aa73451a2a4c2095cbe556ca036839` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:55.832130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03977f-5bc4-44b5-8ea6-94a2c162a84f admin admin] RBAC: Authorizing `identity:delete_user(user_id=60aa73451a2a4c2095cbe556ca036839)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:55.835699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03977f-5bc4-44b5-8ea6-94a2c162a84f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:56.147721 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:56.151764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4e03977f-5bc4-44b5-8ea6-94a2c162a84f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60aa73451a2a4c2095cbe556ca036839'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:56.161044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4e03977f-5bc4-44b5-8ea6-94a2c162a84f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60aa73451a2a4c2095cbe556ca036839'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:56.161782 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4e03977f-5bc4-44b5-8ea6-94a2c162a84f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60aa73451a2a4c2095cbe556ca036839'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:56.180868 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4e03977f-5bc4-44b5-8ea6-94a2c162a84f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60aa73451a2a4c2095cbe556ca036839'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:56.180868 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4e03977f-5bc4-44b5-8ea6-94a2c162a84f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60aa73451a2a4c2095cbe556ca036839'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:56.259449 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 913/1808] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:06:55 2022] DELETE /identity/v3/users/60aa73451a2a4c2095cbe556ca036839 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:57.074201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:58.012097 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c2f9f-cd14-41fd-9033-4978242a54bd admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:58.013650 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-339c2f9f-cd14-41fd-9033-4978242a54bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:58.031488 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c2f9f-cd14-41fd-9033-4978242a54bd admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:58.090762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:58.113620 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a53299-304d-444f-9aa1-08b780a9e7e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:58.114872 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a53299-304d-444f-9aa1-08b780a9e7e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:58.115969 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a53299-304d-444f-9aa1-08b780a9e7e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:58.116493 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a53299-304d-444f-9aa1-08b780a9e7e5 admin admin] PATH_INFO: `/v3/projects/74a8979a91b14849a76e92d528669888` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:58.118803 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a53299-304d-444f-9aa1-08b780a9e7e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74a8979a91b14849a76e92d528669888)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:58.125393 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a53299-304d-444f-9aa1-08b780a9e7e5 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:58.276848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-f6a53299-304d-444f-9aa1-08b780a9e7e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74a8979a91b14849a76e92d528669888', 'request_id': 'req-f6a53299-304d-444f-9aa1-08b780a9e7e5'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:58.291008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-f6a53299-304d-444f-9aa1-08b780a9e7e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74a8979a91b14849a76e92d528669888', 'request_id': 'req-f6a53299-304d-444f-9aa1-08b780a9e7e5'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:58.305494 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-f6a53299-304d-444f-9aa1-08b780a9e7e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74a8979a91b14849a76e92d528669888', 'request_id': 'req-f6a53299-304d-444f-9aa1-08b780a9e7e5'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:58.451376 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-f6a53299-304d-444f-9aa1-08b780a9e7e5 admin admin] The token cache is being invalidate because project 74a8979a91b14849a76e92d528669888 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:06:58.451376 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-f6a53299-304d-444f-9aa1-08b780a9e7e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:58.451376 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-f6a53299-304d-444f-9aa1-08b780a9e7e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:06:58.451376 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 896/1809] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:06:58 2022] DELETE /identity/v3/projects/74a8979a91b14849a76e92d528669888 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:58.452384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb9aa2-fc19-42f6-b37b-11acdce9e87d None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:58.452667 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958fb27-da7c-4b63-b56e-fbf147cc1e7f None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:58.453311 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958fb27-da7c-4b63-b56e-fbf147cc1e7f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:58.453616 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958fb27-da7c-4b63-b56e-fbf147cc1e7f None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:58.453957 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb9aa2-fc19-42f6-b37b-11acdce9e87d None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:58.454686 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb9aa2-fc19-42f6-b37b-11acdce9e87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:58.457352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 914/1810] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 2 09:06:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:06:58.496061 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-bcbb9aa2-fc19-42f6-b37b-11acdce9e87d None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:58.496198 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e16f40-d3b5-48cb-b58c-94a1ad9f777e None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:58.496198 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e16f40-d3b5-48cb-b58c-94a1ad9f777e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:58.496198 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e16f40-d3b5-48cb-b58c-94a1ad9f777e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:58.496582 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-bcbb9aa2-fc19-42f6-b37b-11acdce9e87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:58.534822 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-07e16f40-d3b5-48cb-b58c-94a1ad9f777e None None] MFA Rules not processed for user `3191d804e8dc4b1a85f490319833ec07`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:58.691371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-07e16f40-d3b5-48cb-b58c-94a1ad9f777e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:58.692984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 897/1811] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:06:58.693583 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 915/1812] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 2 09:06:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:58.695127 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd4205-e710-4dfb-845c-ce2d3e4775cd None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:58.695419 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd4205-e710-4dfb-845c-ce2d3e4775cd None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:58.695699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd4205-e710-4dfb-845c-ce2d3e4775cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:58.719678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-f1fd4205-e710-4dfb-845c-ce2d3e4775cd None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:58.720945 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f1fd4205-e710-4dfb-845c-ce2d3e4775cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:58.726146 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e16f40-d3b5-48cb-b58c-94a1ad9f777e None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:58.727957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-07e16f40-d3b5-48cb-b58c-94a1ad9f777e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:58.732202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 898/1813] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:58.736261 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd4205-e710-4dfb-845c-ce2d3e4775cd None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:58.748433 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd4205-e710-4dfb-845c-ce2d3e4775cd None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:58.754426 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7c85b-3349-4e6c-97e3-d4bcd1d16be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:58.755234 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7c85b-3349-4e6c-97e3-d4bcd1d16be9 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:58.755523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7c85b-3349-4e6c-97e3-d4bcd1d16be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:58.755800 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7c85b-3349-4e6c-97e3-d4bcd1d16be9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:58.756508 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e16f40-d3b5-48cb-b58c-94a1ad9f777e None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:58.756999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba7c85b-3349-4e6c-97e3-d4bcd1d16be9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:06:58.756999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba7c85b-3349-4e6c-97e3-d4bcd1d16be9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:58.758651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba7c85b-3349-4e6c-97e3-d4bcd1d16be9 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:58.772046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c2585-ce9b-4099-9280-08656666c9a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:58.773110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c2585-ce9b-4099-9280-08656666c9a1 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:58.773821 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 899/1814] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:06:58.777593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c2585-ce9b-4099-9280-08656666c9a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:58.778707 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7c85b-3349-4e6c-97e3-d4bcd1d16be9 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:58.779096 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c2585-ce9b-4099-9280-08656666c9a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:58.793636 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7c85b-3349-4e6c-97e3-d4bcd1d16be9 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:58.802595 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c2585-ce9b-4099-9280-08656666c9a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:58.807533 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a7af4-218f-4988-9ed3-99f2696dbb20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:58.807533 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a7af4-218f-4988-9ed3-99f2696dbb20 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:58.807533 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a7af4-218f-4988-9ed3-99f2696dbb20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:58.807533 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a7af4-218f-4988-9ed3-99f2696dbb20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:58.807924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c2585-ce9b-4099-9280-08656666c9a1 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:58.808025 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a7af4-218f-4988-9ed3-99f2696dbb20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:58.810864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a7af4-218f-4988-9ed3-99f2696dbb20 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:58.841723 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 916/1815] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:06:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:06:59.000490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 900/1816] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:06:58 2022] POST /identity/v3/projects => generated 385 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:06:59.001367 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c2585-ce9b-4099-9280-08656666c9a1 service nova] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:59.015835 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c2585-ce9b-4099-9280-08656666c9a1 service nova] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:59.109286 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae4f46-ea70-4209-91bb-b29ee16a60e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:59.114492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae4f46-ea70-4209-91bb-b29ee16a60e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:59.114860 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae4f46-ea70-4209-91bb-b29ee16a60e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:59.115408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae4f46-ea70-4209-91bb-b29ee16a60e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:59.116802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae4f46-ea70-4209-91bb-b29ee16a60e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:59.119002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae4f46-ea70-4209-91bb-b29ee16a60e0 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:59.258645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:06:59.275036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a7af4-218f-4988-9ed3-99f2696dbb20 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:59.280790 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 917/1817] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:06:58 2022] POST /identity/v3/users => generated 293 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:06:59.300785 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a7af4-218f-4988-9ed3-99f2696dbb20 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:59.312401 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a07d12-363e-4a1f-9b15-bebe640f0578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:59.313061 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a07d12-363e-4a1f-9b15-bebe640f0578 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:59.313342 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a07d12-363e-4a1f-9b15-bebe640f0578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:59.313605 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a07d12-363e-4a1f-9b15-bebe640f0578 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:59.314375 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a07d12-363e-4a1f-9b15-bebe640f0578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:59.316044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a07d12-363e-4a1f-9b15-bebe640f0578 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:59.332541 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 901/1818] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:59.336695 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae4f46-ea70-4209-91bb-b29ee16a60e0 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:59.350811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae4f46-ea70-4209-91bb-b29ee16a60e0 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:59.360328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d67f53c-8ab2-4095-9d19-3e12a160b3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:59.360328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67f53c-8ab2-4095-9d19-3e12a160b3ed admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:59.360328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67f53c-8ab2-4095-9d19-3e12a160b3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:59.360328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d67f53c-8ab2-4095-9d19-3e12a160b3ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:59.360842 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d67f53c-8ab2-4095-9d19-3e12a160b3ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:59.361397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d67f53c-8ab2-4095-9d19-3e12a160b3ed admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:59.397308 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 918/1819] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:06:59.398695 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a07d12-363e-4a1f-9b15-bebe640f0578 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:59.417025 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a07d12-363e-4a1f-9b15-bebe640f0578 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:59.424984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3a943-435f-4af4-b662-dda99b9d2ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:59.426078 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3a943-435f-4af4-b662-dda99b9d2ac0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:59.430066 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3a943-435f-4af4-b662-dda99b9d2ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:59.430066 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3a943-435f-4af4-b662-dda99b9d2ac0 admin admin] PATH_INFO: `/v3/projects/314c3737d050430385c0d3ee9435d27b/users/f21317fa0d914128b1246f3015991b15/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:59.452985 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3a943-435f-4af4-b662-dda99b9d2ac0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=314c3737d050430385c0d3ee9435d27b, user_id=f21317fa0d914128b1246f3015991b15, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:59.458418 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3a943-435f-4af4-b662-dda99b9d2ac0 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:06:59.573678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 902/1820] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:06:59 2022] PUT /identity/v3/projects/314c3737d050430385c0d3ee9435d27b/users/f21317fa0d914128b1246f3015991b15/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:06:59.584354 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273adaf7-1809-4361-815f-402d54229085 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:59.584939 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273adaf7-1809-4361-815f-402d54229085 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:59.584939 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273adaf7-1809-4361-815f-402d54229085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:59.622967 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-273adaf7-1809-4361-815f-402d54229085 None None] MFA Rules not processed for user `f21317fa0d914128b1246f3015991b15`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:06:59.624610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-273adaf7-1809-4361-815f-402d54229085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:06:59.748889 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 919/1821] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:06:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:06:59.754515 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3a943-435f-4af4-b662-dda99b9d2ac0 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:06:59.927973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3a943-435f-4af4-b662-dda99b9d2ac0 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:06:59.936760 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7d149-caef-44b9-ad36-1b7270a72fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:06:59.938156 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7d149-caef-44b9-ad36-1b7270a72fe0 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:06:59.940034 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7d149-caef-44b9-ad36-1b7270a72fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:06:59.940584 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7d149-caef-44b9-ad36-1b7270a72fe0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:06:59.941791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7d149-caef-44b9-ad36-1b7270a72fe0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:06:59.945763 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7d149-caef-44b9-ad36-1b7270a72fe0 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:00.161563 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 903/1822] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:06:59 2022] POST /identity/v3/projects => generated 387 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:00.175840 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273adaf7-1809-4361-815f-402d54229085 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:00.212727 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273adaf7-1809-4361-815f-402d54229085 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:00.337638 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6577c-64d2-4316-ba6a-ff92ef8522df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:00.339495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6577c-64d2-4316-ba6a-ff92ef8522df admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:00.339846 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6577c-64d2-4316-ba6a-ff92ef8522df admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:00.340354 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee6577c-64d2-4316-ba6a-ff92ef8522df admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:00.342756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6577c-64d2-4316-ba6a-ff92ef8522df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:00.352112 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee6577c-64d2-4316-ba6a-ff92ef8522df admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:00.389442 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7d149-caef-44b9-ad36-1b7270a72fe0 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:00.394257 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-31c7d149-caef-44b9-ad36-1b7270a72fe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:00.424000 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7d149-caef-44b9-ad36-1b7270a72fe0 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:00.437343 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc0bb7-27ca-4b1f-902a-47144313607a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:00.438407 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc0bb7-27ca-4b1f-902a-47144313607a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:00.438617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc0bb7-27ca-4b1f-902a-47144313607a admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:00.439006 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc0bb7-27ca-4b1f-902a-47144313607a admin admin] PATH_INFO: `/v3/users/05989a39c40b4a2d820c73ad17edd2e9` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:00.443392 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc0bb7-27ca-4b1f-902a-47144313607a admin admin] RBAC: Authorizing `identity:delete_user(user_id=05989a39c40b4a2d820c73ad17edd2e9)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:00.445550 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc0bb7-27ca-4b1f-902a-47144313607a admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:00.513363 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 920/1823] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:07:00 2022] POST /identity/v3/users => generated 300 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:00.518544 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6577c-64d2-4316-ba6a-ff92ef8522df admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:00.536905 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee6577c-64d2-4316-ba6a-ff92ef8522df admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:00.547018 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6900b9e1-9d91-472f-bfd2-a18c4214b66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:00.548217 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900b9e1-9d91-472f-bfd2-a18c4214b66e admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:00.548817 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900b9e1-9d91-472f-bfd2-a18c4214b66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:00.549390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900b9e1-9d91-472f-bfd2-a18c4214b66e admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:00.550742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6900b9e1-9d91-472f-bfd2-a18c4214b66e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:00.553608 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6900b9e1-9d91-472f-bfd2-a18c4214b66e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:00.586208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 921/1824] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:00.588766 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6900b9e1-9d91-472f-bfd2-a18c4214b66e admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:00.637747 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6900b9e1-9d91-472f-bfd2-a18c4214b66e admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:00.637747 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8f5b9-447b-4e2a-8dc7-75b16eef181f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:00.637747 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8f5b9-447b-4e2a-8dc7-75b16eef181f admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:00.637747 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8f5b9-447b-4e2a-8dc7-75b16eef181f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:00.638511 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8f5b9-447b-4e2a-8dc7-75b16eef181f admin admin] PATH_INFO: `/v3/projects/399ae69cf0f8418cba7e9e38e66466fd/users/47d8e698d8ff4ea287b6a9f424c5f16e/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:00.667831 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8f5b9-447b-4e2a-8dc7-75b16eef181f admin admin] RBAC: Authorizing `identity:create_grant(project_id=399ae69cf0f8418cba7e9e38e66466fd, user_id=47d8e698d8ff4ea287b6a9f424c5f16e, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:00.675441 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8f5b9-447b-4e2a-8dc7-75b16eef181f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:00.802162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-befc0bb7-27ca-4b1f-902a-47144313607a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05989a39c40b4a2d820c73ad17edd2e9'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:00.811976 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-befc0bb7-27ca-4b1f-902a-47144313607a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05989a39c40b4a2d820c73ad17edd2e9'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:00.811976 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-befc0bb7-27ca-4b1f-902a-47144313607a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05989a39c40b4a2d820c73ad17edd2e9'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:00.815651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 922/1825] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:00 2022] PUT /identity/v3/projects/399ae69cf0f8418cba7e9e38e66466fd/users/47d8e698d8ff4ea287b6a9f424c5f16e/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:07:00.820745 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8f5b9-447b-4e2a-8dc7-75b16eef181f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:00.823169 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-cad8f5b9-447b-4e2a-8dc7-75b16eef181f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:00.936194 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-befc0bb7-27ca-4b1f-902a-47144313607a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05989a39c40b4a2d820c73ad17edd2e9'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:00.955879 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8f5b9-447b-4e2a-8dc7-75b16eef181f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:00.959432 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-befc0bb7-27ca-4b1f-902a-47144313607a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05989a39c40b4a2d820c73ad17edd2e9'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:00.962659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c054d03-7156-4417-9038-2fc866574e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:00.963945 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c054d03-7156-4417-9038-2fc866574e69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:00.964233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c054d03-7156-4417-9038-2fc866574e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:00.964519 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c054d03-7156-4417-9038-2fc866574e69 admin admin] PATH_INFO: `/v3/users/3d78578a400649b6a866444e4ccb913a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:00.965784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c054d03-7156-4417-9038-2fc866574e69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d78578a400649b6a866444e4ccb913a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:00.968316 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c054d03-7156-4417-9038-2fc866574e69 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:00.971121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 904/1826] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:07:00 2022] DELETE /identity/v3/users/05989a39c40b4a2d820c73ad17edd2e9 => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:00.972438 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc0bb7-27ca-4b1f-902a-47144313607a admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:00.975598 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-befc0bb7-27ca-4b1f-902a-47144313607a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:01.102016 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc0bb7-27ca-4b1f-902a-47144313607a admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:01.107597 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08374b-1f28-403f-a3c4-2500c6f6d716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:01.108782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08374b-1f28-403f-a3c4-2500c6f6d716 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:01.111783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08374b-1f28-403f-a3c4-2500c6f6d716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:01.111783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08374b-1f28-403f-a3c4-2500c6f6d716 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:01.111783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08374b-1f28-403f-a3c4-2500c6f6d716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:01.114362 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08374b-1f28-403f-a3c4-2500c6f6d716 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:01.137332 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 905/1827] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:00 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:01.139981 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f851a12-5b2b-4f4b-9568-2d0515a13415 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:01.139981 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f851a12-5b2b-4f4b-9568-2d0515a13415 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:01.140913 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f851a12-5b2b-4f4b-9568-2d0515a13415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:01.162627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:01.171006 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-4f851a12-5b2b-4f4b-9568-2d0515a13415 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:01.174649 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4f851a12-5b2b-4f4b-9568-2d0515a13415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:01.199759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 906/1828] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:01.199759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f851a12-5b2b-4f4b-9568-2d0515a13415 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:01.242121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f851a12-5b2b-4f4b-9568-2d0515a13415 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:01.256179 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5305b-69c5-4362-8e13-e1ff532dd842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:01.257429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5305b-69c5-4362-8e13-e1ff532dd842 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:01.257742 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5305b-69c5-4362-8e13-e1ff532dd842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:01.258015 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5305b-69c5-4362-8e13-e1ff532dd842 admin admin] PATH_INFO: `/v3/projects/399ae69cf0f8418cba7e9e38e66466fd/users/47d8e698d8ff4ea287b6a9f424c5f16e/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:01.260462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5305b-69c5-4362-8e13-e1ff532dd842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=399ae69cf0f8418cba7e9e38e66466fd, user_id=47d8e698d8ff4ea287b6a9f424c5f16e, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:01.263376 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5305b-69c5-4362-8e13-e1ff532dd842 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:01.278105 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 907/1829] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:01 2022] PUT /identity/v3/projects/399ae69cf0f8418cba7e9e38e66466fd/users/47d8e698d8ff4ea287b6a9f424c5f16e/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:01.283121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5305b-69c5-4362-8e13-e1ff532dd842 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:01.290366 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-6ba5305b-69c5-4362-8e13-e1ff532dd842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:01.296687 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:01.333421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5c054d03-7156-4417-9038-2fc866574e69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d78578a400649b6a866444e4ccb913a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:01.351440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5c054d03-7156-4417-9038-2fc866574e69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d78578a400649b6a866444e4ccb913a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:01.352249 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5c054d03-7156-4417-9038-2fc866574e69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d78578a400649b6a866444e4ccb913a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:01.376739 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5c054d03-7156-4417-9038-2fc866574e69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d78578a400649b6a866444e4ccb913a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:01.380018 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5c054d03-7156-4417-9038-2fc866574e69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d78578a400649b6a866444e4ccb913a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:01.427870 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5305b-69c5-4362-8e13-e1ff532dd842 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:01.431772 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad03287-58f3-4733-8219-7942fc1d23bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:01.433300 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad03287-58f3-4733-8219-7942fc1d23bf service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:01.433587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad03287-58f3-4733-8219-7942fc1d23bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:01.433866 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad03287-58f3-4733-8219-7942fc1d23bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:01.435878 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4ad03287-58f3-4733-8219-7942fc1d23bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:01.507323 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 923/1830] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:07:00 2022] DELETE /identity/v3/users/3d78578a400649b6a866444e4ccb913a => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:01.509511 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63804e28-4e11-4447-888f-9d5e97ae6ad2 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:01.515147 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63804e28-4e11-4447-888f-9d5e97ae6ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:01.515147 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63804e28-4e11-4447-888f-9d5e97ae6ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:01.526303 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad03287-58f3-4733-8219-7942fc1d23bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:01.529068 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad03287-58f3-4733-8219-7942fc1d23bf service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:01.547479 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 908/1831] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:07:01.549732 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e406df-bd8e-4eb1-bfc2-18e4d82eafbf None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:01.550232 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e406df-bd8e-4eb1-bfc2-18e4d82eafbf None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:01.550687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e406df-bd8e-4eb1-bfc2-18e4d82eafbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:01.568004 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-63804e28-4e11-4447-888f-9d5e97ae6ad2 None None] MFA Rules not processed for user `47d8e698d8ff4ea287b6a9f424c5f16e`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:01.568004 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-63804e28-4e11-4447-888f-9d5e97ae6ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:01.582914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-c5e406df-bd8e-4eb1-bfc2-18e4d82eafbf None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:01.584169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-c5e406df-bd8e-4eb1-bfc2-18e4d82eafbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:01.685249 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 924/1832] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:01 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:01.700947 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 909/1833] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:01.711841 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63804e28-4e11-4447-888f-9d5e97ae6ad2 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:01.714748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-63804e28-4e11-4447-888f-9d5e97ae6ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:01.715993 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e406df-bd8e-4eb1-bfc2-18e4d82eafbf None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:01.717590 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-c5e406df-bd8e-4eb1-bfc2-18e4d82eafbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:01.790892 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e406df-bd8e-4eb1-bfc2-18e4d82eafbf None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:01.796589 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e8189-7d70-4f85-afe4-128cd06a5f8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:01.797499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e8189-7d70-4f85-afe4-128cd06a5f8a service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:01.798015 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e8189-7d70-4f85-afe4-128cd06a5f8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:01.798399 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e8189-7d70-4f85-afe4-128cd06a5f8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:01.811146 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63804e28-4e11-4447-888f-9d5e97ae6ad2 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:01.812676 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e8189-7d70-4f85-afe4-128cd06a5f8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:01.814813 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e8189-7d70-4f85-afe4-128cd06a5f8a service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:01.816475 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b85b44-3780-49aa-b638-d5d4a9c52417 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:01.817113 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b85b44-3780-49aa-b638-d5d4a9c52417 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:01.817402 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b85b44-3780-49aa-b638-d5d4a9c52417 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:01.817685 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b85b44-3780-49aa-b638-d5d4a9c52417 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:01.832650 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 910/1834] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:07:01.839875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b85b44-3780-49aa-b638-d5d4a9c52417 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:01.841754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b85b44-3780-49aa-b638-d5d4a9c52417 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:01.864508 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 925/1835] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:01 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:07:02.218147 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e8189-7d70-4f85-afe4-128cd06a5f8a service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:02.218147 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f90e8189-7d70-4f85-afe4-128cd06a5f8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:02.240108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e8189-7d70-4f85-afe4-128cd06a5f8a service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:02.246063 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e39bfd-f41c-4229-8175-0b4fcf368c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:02.247044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e39bfd-f41c-4229-8175-0b4fcf368c22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:02.248050 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e39bfd-f41c-4229-8175-0b4fcf368c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:02.252996 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e39bfd-f41c-4229-8175-0b4fcf368c22 admin admin] PATH_INFO: `/v3/projects/6956616275cd4ebfa23f551e04cc4710` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:02.259425 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e39bfd-f41c-4229-8175-0b4fcf368c22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6956616275cd4ebfa23f551e04cc4710)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:02.261168 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e39bfd-f41c-4229-8175-0b4fcf368c22 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:02.390706 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-36e39bfd-f41c-4229-8175-0b4fcf368c22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6956616275cd4ebfa23f551e04cc4710', 'request_id': 'req-36e39bfd-f41c-4229-8175-0b4fcf368c22'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:02.399751 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-36e39bfd-f41c-4229-8175-0b4fcf368c22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6956616275cd4ebfa23f551e04cc4710', 'request_id': 'req-36e39bfd-f41c-4229-8175-0b4fcf368c22'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:02.406849 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-36e39bfd-f41c-4229-8175-0b4fcf368c22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6956616275cd4ebfa23f551e04cc4710', 'request_id': 'req-36e39bfd-f41c-4229-8175-0b4fcf368c22'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:02.498026 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-36e39bfd-f41c-4229-8175-0b4fcf368c22 admin admin] The token cache is being invalidate because project 6956616275cd4ebfa23f551e04cc4710 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:07:02.498026 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-36e39bfd-f41c-4229-8175-0b4fcf368c22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:02.500755 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-36e39bfd-f41c-4229-8175-0b4fcf368c22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:02.502447 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 911/1836] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:07:02 2022] DELETE /identity/v3/projects/6956616275cd4ebfa23f551e04cc4710 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:02.509332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b85b44-3780-49aa-b638-d5d4a9c52417 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:02.522309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b85b44-3780-49aa-b638-d5d4a9c52417 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:02.545971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e39bfd-f41c-4229-8175-0b4fcf368c22 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:02.548335 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-36e39bfd-f41c-4229-8175-0b4fcf368c22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:02.624612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649df3c5-f7f7-4baf-945a-e10c1a65efd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:02.625821 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649df3c5-f7f7-4baf-945a-e10c1a65efd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:02.628503 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649df3c5-f7f7-4baf-945a-e10c1a65efd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:02.628503 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649df3c5-f7f7-4baf-945a-e10c1a65efd6 admin admin] PATH_INFO: `/v3/users/81e41fe5ac9b4b2abc54b21942a934d9` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:02.628503 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649df3c5-f7f7-4baf-945a-e10c1a65efd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81e41fe5ac9b4b2abc54b21942a934d9)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:02.630726 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649df3c5-f7f7-4baf-945a-e10c1a65efd6 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:02.660330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e39bfd-f41c-4229-8175-0b4fcf368c22 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:02.665745 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9e4c3-800b-47d6-a1ef-f435228c1a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:02.666527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9e4c3-800b-47d6-a1ef-f435228c1a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:02.666794 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9e4c3-800b-47d6-a1ef-f435228c1a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:02.667739 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9e4c3-800b-47d6-a1ef-f435228c1a2e admin admin] PATH_INFO: `/v3/projects/2faa2bc3068849959e87189502f18f07` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:02.668319 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9e4c3-800b-47d6-a1ef-f435228c1a2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2faa2bc3068849959e87189502f18f07)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:02.670313 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f9e4c3-800b-47d6-a1ef-f435228c1a2e admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:03.099840 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-64f9e4c3-800b-47d6-a1ef-f435228c1a2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2faa2bc3068849959e87189502f18f07', 'request_id': 'req-64f9e4c3-800b-47d6-a1ef-f435228c1a2e'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:03.107578 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-64f9e4c3-800b-47d6-a1ef-f435228c1a2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2faa2bc3068849959e87189502f18f07', 'request_id': 'req-64f9e4c3-800b-47d6-a1ef-f435228c1a2e'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:03.116736 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-64f9e4c3-800b-47d6-a1ef-f435228c1a2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2faa2bc3068849959e87189502f18f07', 'request_id': 'req-64f9e4c3-800b-47d6-a1ef-f435228c1a2e'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:03.117519 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-649df3c5-f7f7-4baf-945a-e10c1a65efd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81e41fe5ac9b4b2abc54b21942a934d9'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:03.131196 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-649df3c5-f7f7-4baf-945a-e10c1a65efd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81e41fe5ac9b4b2abc54b21942a934d9'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:03.131196 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-649df3c5-f7f7-4baf-945a-e10c1a65efd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81e41fe5ac9b4b2abc54b21942a934d9'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:03.145650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-649df3c5-f7f7-4baf-945a-e10c1a65efd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81e41fe5ac9b4b2abc54b21942a934d9'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:03.147352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-649df3c5-f7f7-4baf-945a-e10c1a65efd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81e41fe5ac9b4b2abc54b21942a934d9'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:03.302283 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-64f9e4c3-800b-47d6-a1ef-f435228c1a2e admin admin] The token cache is being invalidate because project 2faa2bc3068849959e87189502f18f07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:07:03.302602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-64f9e4c3-800b-47d6-a1ef-f435228c1a2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:03.303969 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-64f9e4c3-800b-47d6-a1ef-f435228c1a2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:03.310964 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 912/1837] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:07:02 2022] DELETE /identity/v3/projects/2faa2bc3068849959e87189502f18f07 => generated 0 bytes in 763 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:07:03.312762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9e4c3-800b-47d6-a1ef-f435228c1a2e admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:03.314659 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-64f9e4c3-800b-47d6-a1ef-f435228c1a2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:03.377487 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 926/1838] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:07:02 2022] DELETE /identity/v3/users/81e41fe5ac9b4b2abc54b21942a934d9 => generated 0 bytes in 869 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:03.379037 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17191289-a64a-4164-8d05-d18819ed4001 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:03.379326 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17191289-a64a-4164-8d05-d18819ed4001 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:03.379605 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17191289-a64a-4164-8d05-d18819ed4001 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:03.382968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 927/1839] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 2 09:07:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:07:03.390120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b88350-b839-4285-9bf3-2c870a47b8ad None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:03.390487 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b88350-b839-4285-9bf3-2c870a47b8ad None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:03.390670 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b88350-b839-4285-9bf3-2c870a47b8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:03.423120 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9e4c3-800b-47d6-a1ef-f435228c1a2e admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:03.425358 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-35b88350-b839-4285-9bf3-2c870a47b8ad None None] MFA Rules not processed for user `3191d804e8dc4b1a85f490319833ec07`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:03.426580 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-35b88350-b839-4285-9bf3-2c870a47b8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:03.541761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1eef6-de4e-4414-8a3c-c369ff67ddeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:03.543210 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1eef6-de4e-4414-8a3c-c369ff67ddeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:03.543663 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1eef6-de4e-4414-8a3c-c369ff67ddeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:03.544132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f1eef6-de4e-4414-8a3c-c369ff67ddeb admin admin] PATH_INFO: `/v3/users/da3a0adb3761484bb1e7a852461a448a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:03.545684 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1eef6-de4e-4414-8a3c-c369ff67ddeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=da3a0adb3761484bb1e7a852461a448a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:03.550677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f1eef6-de4e-4414-8a3c-c369ff67ddeb admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:03.611802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 928/1840] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 2 09:07:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:03.625070 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b88350-b839-4285-9bf3-2c870a47b8ad None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:03.626834 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-35b88350-b839-4285-9bf3-2c870a47b8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:03.643694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b88350-b839-4285-9bf3-2c870a47b8ad None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:03.649827 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d32e2c-bad6-40d7-a78e-e144f01a53fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:03.650513 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d32e2c-bad6-40d7-a78e-e144f01a53fa service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:03.650799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d32e2c-bad6-40d7-a78e-e144f01a53fa service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:03.654109 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d32e2c-bad6-40d7-a78e-e144f01a53fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:03.674941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d32e2c-bad6-40d7-a78e-e144f01a53fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:03.681661 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d32e2c-bad6-40d7-a78e-e144f01a53fa service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:03.694618 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 929/1841] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:07:03.742751 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:03.803612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d32e2c-bad6-40d7-a78e-e144f01a53fa service nova] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:03.809974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-83d32e2c-bad6-40d7-a78e-e144f01a53fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:03.827219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d32e2c-bad6-40d7-a78e-e144f01a53fa service nova] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:03.833051 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ebff56-bc6d-48cc-87ea-8980033895e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:03.833732 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebff56-bc6d-48cc-87ea-8980033895e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:03.834610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebff56-bc6d-48cc-87ea-8980033895e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:03.835046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ebff56-bc6d-48cc-87ea-8980033895e6 admin admin] PATH_INFO: `/v3/users/f21317fa0d914128b1246f3015991b15` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:03.836415 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ebff56-bc6d-48cc-87ea-8980033895e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f21317fa0d914128b1246f3015991b15)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:03.839205 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ebff56-bc6d-48cc-87ea-8980033895e6 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:04.074146 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:05.060118 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-c3f1eef6-de4e-4414-8a3c-c369ff67ddeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da3a0adb3761484bb1e7a852461a448a'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:05.069431 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-c3f1eef6-de4e-4414-8a3c-c369ff67ddeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da3a0adb3761484bb1e7a852461a448a'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:05.070384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-c3f1eef6-de4e-4414-8a3c-c369ff67ddeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da3a0adb3761484bb1e7a852461a448a'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:05.349104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-03ebff56-bc6d-48cc-87ea-8980033895e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f21317fa0d914128b1246f3015991b15'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:05.357360 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-03ebff56-bc6d-48cc-87ea-8980033895e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f21317fa0d914128b1246f3015991b15'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:05.358206 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-03ebff56-bc6d-48cc-87ea-8980033895e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f21317fa0d914128b1246f3015991b15'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:05.370534 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-c3f1eef6-de4e-4414-8a3c-c369ff67ddeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da3a0adb3761484bb1e7a852461a448a'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:05.374177 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-03ebff56-bc6d-48cc-87ea-8980033895e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f21317fa0d914128b1246f3015991b15'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:05.374177 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-03ebff56-bc6d-48cc-87ea-8980033895e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f21317fa0d914128b1246f3015991b15'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:05.387695 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-c3f1eef6-de4e-4414-8a3c-c369ff67ddeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da3a0adb3761484bb1e7a852461a448a'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:05.396629 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 913/1842] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:07:03 2022] DELETE /identity/v3/users/da3a0adb3761484bb1e7a852461a448a => generated 0 bytes in 2085 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:05.399191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1eef6-de4e-4414-8a3c-c369ff67ddeb admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:05.399721 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-c3f1eef6-de4e-4414-8a3c-c369ff67ddeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:05.487937 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 930/1843] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:07:03 2022] DELETE /identity/v3/users/f21317fa0d914128b1246f3015991b15 => generated 0 bytes in 1682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:05.491274 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcc757-1e1c-40ea-938c-fb1a54067fa1 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:05.491741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcc757-1e1c-40ea-938c-fb1a54067fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:05.492082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dcc757-1e1c-40ea-938c-fb1a54067fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:05.519967 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-95dcc757-1e1c-40ea-938c-fb1a54067fa1 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:05.521445 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-95dcc757-1e1c-40ea-938c-fb1a54067fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:05.533339 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f1eef6-de4e-4414-8a3c-c369ff67ddeb admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:05.619865 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d4c8d-1530-4412-a0e1-f4435208e6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:05.620804 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d4c8d-1530-4412-a0e1-f4435208e6ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:05.621093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d4c8d-1530-4412-a0e1-f4435208e6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:05.621431 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4d4c8d-1530-4412-a0e1-f4435208e6ad admin admin] PATH_INFO: `/v3/projects/85c2adc914454a7f86b6ec24754a822e` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:05.623402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d4c8d-1530-4412-a0e1-f4435208e6ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=85c2adc914454a7f86b6ec24754a822e)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:05.627269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4d4c8d-1530-4412-a0e1-f4435208e6ad admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:05.693954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 931/1844] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:07:05.752596 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dcc757-1e1c-40ea-938c-fb1a54067fa1 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:05.754931 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-95dcc757-1e1c-40ea-938c-fb1a54067fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:05.835220 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-6e4d4c8d-1530-4412-a0e1-f4435208e6ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85c2adc914454a7f86b6ec24754a822e', 'request_id': 'req-6e4d4c8d-1530-4412-a0e1-f4435208e6ad'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:05.852166 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dcc757-1e1c-40ea-938c-fb1a54067fa1 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:05.875095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-6e4d4c8d-1530-4412-a0e1-f4435208e6ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85c2adc914454a7f86b6ec24754a822e', 'request_id': 'req-6e4d4c8d-1530-4412-a0e1-f4435208e6ad'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:05.901316 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-6e4d4c8d-1530-4412-a0e1-f4435208e6ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85c2adc914454a7f86b6ec24754a822e', 'request_id': 'req-6e4d4c8d-1530-4412-a0e1-f4435208e6ad'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:05.915768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f663c5-78a5-4f49-9f8c-745dbdbb21cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:05.917190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f663c5-78a5-4f49-9f8c-745dbdbb21cf service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:05.917543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f663c5-78a5-4f49-9f8c-745dbdbb21cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:05.917908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f663c5-78a5-4f49-9f8c-745dbdbb21cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:06.074449 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f663c5-78a5-4f49-9f8c-745dbdbb21cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:06.075207 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-6e4d4c8d-1530-4412-a0e1-f4435208e6ad admin admin] The token cache is being invalidate because project 85c2adc914454a7f86b6ec24754a822e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:07:06.076691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-6e4d4c8d-1530-4412-a0e1-f4435208e6ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:06.078100 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f663c5-78a5-4f49-9f8c-745dbdbb21cf service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:06.078252 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-6e4d4c8d-1530-4412-a0e1-f4435208e6ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:06.083474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 914/1845] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:07:05 2022] DELETE /identity/v3/projects/85c2adc914454a7f86b6ec24754a822e => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:06.084876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d4c8d-1530-4412-a0e1-f4435208e6ad admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:06.086674 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-6e4d4c8d-1530-4412-a0e1-f4435208e6ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:06.099844 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 932/1846] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:07:06.102427 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4d4c8d-1530-4412-a0e1-f4435208e6ad admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:06.109171 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11462fd3-85aa-4847-8ad7-2172113d8609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:06.110276 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11462fd3-85aa-4847-8ad7-2172113d8609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:06.110640 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11462fd3-85aa-4847-8ad7-2172113d8609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:06.110988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11462fd3-85aa-4847-8ad7-2172113d8609 admin admin] PATH_INFO: `/v3/users/3d033cd703c342809e214aef8fa6aa72` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:06.112937 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11462fd3-85aa-4847-8ad7-2172113d8609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d033cd703c342809e214aef8fa6aa72)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:06.115327 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11462fd3-85aa-4847-8ad7-2172113d8609 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:06.424579 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-11462fd3-85aa-4847-8ad7-2172113d8609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d033cd703c342809e214aef8fa6aa72'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:06.441041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-11462fd3-85aa-4847-8ad7-2172113d8609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d033cd703c342809e214aef8fa6aa72'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:06.441928 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-11462fd3-85aa-4847-8ad7-2172113d8609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d033cd703c342809e214aef8fa6aa72'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:06.651471 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-11462fd3-85aa-4847-8ad7-2172113d8609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d033cd703c342809e214aef8fa6aa72'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:06.674733 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f663c5-78a5-4f49-9f8c-745dbdbb21cf service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:06.678831 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b3f663c5-78a5-4f49-9f8c-745dbdbb21cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:06.679678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-11462fd3-85aa-4847-8ad7-2172113d8609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d033cd703c342809e214aef8fa6aa72'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:06.689807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 915/1847] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:07:06 2022] DELETE /identity/v3/users/3d033cd703c342809e214aef8fa6aa72 => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:06.695677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f663c5-78a5-4f49-9f8c-745dbdbb21cf service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:06.785414 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e2792-7b7a-419b-a38b-6a0c72717726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:06.786950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e2792-7b7a-419b-a38b-6a0c72717726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:06.787600 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e2792-7b7a-419b-a38b-6a0c72717726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:06.788265 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e2792-7b7a-419b-a38b-6a0c72717726 admin admin] PATH_INFO: `/v3/projects/2469c01d9a7e4c93ad62310367eff906` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:06.790272 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e2792-7b7a-419b-a38b-6a0c72717726 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2469c01d9a7e4c93ad62310367eff906)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:06.793827 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06e2792-7b7a-419b-a38b-6a0c72717726 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:06.993850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f06e2792-7b7a-419b-a38b-6a0c72717726 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2469c01d9a7e4c93ad62310367eff906', 'request_id': 'req-f06e2792-7b7a-419b-a38b-6a0c72717726'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:07.005437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f06e2792-7b7a-419b-a38b-6a0c72717726 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2469c01d9a7e4c93ad62310367eff906', 'request_id': 'req-f06e2792-7b7a-419b-a38b-6a0c72717726'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:07.085725 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11462fd3-85aa-4847-8ad7-2172113d8609 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:07.088475 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-11462fd3-85aa-4847-8ad7-2172113d8609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:07.102151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f06e2792-7b7a-419b-a38b-6a0c72717726 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2469c01d9a7e4c93ad62310367eff906', 'request_id': 'req-f06e2792-7b7a-419b-a38b-6a0c72717726'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:07.118917 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11462fd3-85aa-4847-8ad7-2172113d8609 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:07.131506 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f06e2792-7b7a-419b-a38b-6a0c72717726 admin admin] The token cache is being invalidate because project 2469c01d9a7e4c93ad62310367eff906 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:07:07.131843 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f06e2792-7b7a-419b-a38b-6a0c72717726 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:07.133168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f06e2792-7b7a-419b-a38b-6a0c72717726 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:07.139454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 933/1848] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:07:06 2022] DELETE /identity/v3/projects/2469c01d9a7e4c93ad62310367eff906 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:07.141237 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094f332-b6bb-4eed-a93e-d18df22427f9 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:07.141622 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094f332-b6bb-4eed-a93e-d18df22427f9 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:07.141925 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094f332-b6bb-4eed-a93e-d18df22427f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:07.165625 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9094f332-b6bb-4eed-a93e-d18df22427f9 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:07.167788 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9094f332-b6bb-4eed-a93e-d18df22427f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:07.228417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b7576-535b-4fe8-a782-06f0b854ce55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:07.230504 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b7576-535b-4fe8-a782-06f0b854ce55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:07.232450 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b7576-535b-4fe8-a782-06f0b854ce55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:07.233716 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b7576-535b-4fe8-a782-06f0b854ce55 admin admin] PATH_INFO: `/v3/projects/314c3737d050430385c0d3ee9435d27b` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:07.236010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b7576-535b-4fe8-a782-06f0b854ce55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=314c3737d050430385c0d3ee9435d27b)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:07.239429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b7576-535b-4fe8-a782-06f0b854ce55 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:07.333115 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 934/1849] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:07.355742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bada4-008b-4e50-a1aa-5b339f8882a0 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:07.356061 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bada4-008b-4e50-a1aa-5b339f8882a0 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:07.356327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bada4-008b-4e50-a1aa-5b339f8882a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:07.381875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-eb8bada4-008b-4e50-a1aa-5b339f8882a0 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:07.381875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-eb8bada4-008b-4e50-a1aa-5b339f8882a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:07.398953 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 935/1850] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:07.401913 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bada4-008b-4e50-a1aa-5b339f8882a0 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:07.419388 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bada4-008b-4e50-a1aa-5b339f8882a0 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:07.468593 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-436b7576-535b-4fe8-a782-06f0b854ce55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '314c3737d050430385c0d3ee9435d27b', 'request_id': 'req-436b7576-535b-4fe8-a782-06f0b854ce55'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:07.484305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-436b7576-535b-4fe8-a782-06f0b854ce55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '314c3737d050430385c0d3ee9435d27b', 'request_id': 'req-436b7576-535b-4fe8-a782-06f0b854ce55'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:07.492675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:07.502170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-436b7576-535b-4fe8-a782-06f0b854ce55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '314c3737d050430385c0d3ee9435d27b', 'request_id': 'req-436b7576-535b-4fe8-a782-06f0b854ce55'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:07.560819 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c1e42-81c8-4de7-9523-cd35718b9912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:07.561550 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c1e42-81c8-4de7-9523-cd35718b9912 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:07.561835 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c1e42-81c8-4de7-9523-cd35718b9912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:07.563452 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c1e42-81c8-4de7-9523-cd35718b9912 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:07.566865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c1e42-81c8-4de7-9523-cd35718b9912 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:07:07.567247 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c1e42-81c8-4de7-9523-cd35718b9912 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:07.571814 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c1e42-81c8-4de7-9523-cd35718b9912 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:07.608782 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 936/1851] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:07:07.614035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c1e42-81c8-4de7-9523-cd35718b9912 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:07.632392 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c1e42-81c8-4de7-9523-cd35718b9912 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:07.645610 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-436b7576-535b-4fe8-a782-06f0b854ce55 admin admin] The token cache is being invalidate because project 314c3737d050430385c0d3ee9435d27b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:07:07.646205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-436b7576-535b-4fe8-a782-06f0b854ce55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:07.649103 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd6308-00bf-40a7-88e6-df9c01c14d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:07.650303 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd6308-00bf-40a7-88e6-df9c01c14d9c admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:07.651463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd6308-00bf-40a7-88e6-df9c01c14d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:07.652023 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd6308-00bf-40a7-88e6-df9c01c14d9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:07.652542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-436b7576-535b-4fe8-a782-06f0b854ce55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:07.652873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd6308-00bf-40a7-88e6-df9c01c14d9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:07.656335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd6308-00bf-40a7-88e6-df9c01c14d9c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:07.658314 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 916/1852] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:07:07 2022] DELETE /identity/v3/projects/314c3737d050430385c0d3ee9435d27b => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:07.664356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b7576-535b-4fe8-a782-06f0b854ce55 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:07.669497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-436b7576-535b-4fe8-a782-06f0b854ce55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:07.695653 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b7576-535b-4fe8-a782-06f0b854ce55 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:07.705257 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117a016-0b8b-4b22-8f0e-3757248ece14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:07.706387 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117a016-0b8b-4b22-8f0e-3757248ece14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:07.706826 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117a016-0b8b-4b22-8f0e-3757248ece14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:07.707247 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c117a016-0b8b-4b22-8f0e-3757248ece14 admin admin] PATH_INFO: `/v3/users/47d8e698d8ff4ea287b6a9f424c5f16e` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:07.710139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c117a016-0b8b-4b22-8f0e-3757248ece14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47d8e698d8ff4ea287b6a9f424c5f16e)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:07.714869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c117a016-0b8b-4b22-8f0e-3757248ece14 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:07.757601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 937/1853] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:07:07 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:07:07.763975 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd6308-00bf-40a7-88e6-df9c01c14d9c admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:07.767075 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c9fd6308-00bf-40a7-88e6-df9c01c14d9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:07.887633 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd6308-00bf-40a7-88e6-df9c01c14d9c admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:07.906267 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397bc365-f888-4bea-9ff0-30dfdc5be827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:07.906267 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bc365-f888-4bea-9ff0-30dfdc5be827 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:07.906836 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bc365-f888-4bea-9ff0-30dfdc5be827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:07.907128 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bc365-f888-4bea-9ff0-30dfdc5be827 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:07.907841 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397bc365-f888-4bea-9ff0-30dfdc5be827 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:07.909874 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397bc365-f888-4bea-9ff0-30dfdc5be827 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:08.017752 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-c117a016-0b8b-4b22-8f0e-3757248ece14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47d8e698d8ff4ea287b6a9f424c5f16e'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:08.028665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-c117a016-0b8b-4b22-8f0e-3757248ece14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47d8e698d8ff4ea287b6a9f424c5f16e'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:08.029458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-c117a016-0b8b-4b22-8f0e-3757248ece14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47d8e698d8ff4ea287b6a9f424c5f16e'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:08.037754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 938/1854] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:07:07 2022] POST /identity/v3/users => generated 293 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:08.037754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397bc365-f888-4bea-9ff0-30dfdc5be827 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:08.038971 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-397bc365-f888-4bea-9ff0-30dfdc5be827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:08.052695 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397bc365-f888-4bea-9ff0-30dfdc5be827 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:08.060640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31cb1a-c984-40fd-a81e-cf3f366db8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:08.061591 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31cb1a-c984-40fd-a81e-cf3f366db8e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:08.062063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31cb1a-c984-40fd-a81e-cf3f366db8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:08.062894 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31cb1a-c984-40fd-a81e-cf3f366db8e3 admin admin] PATH_INFO: `/v3/projects/5c527d333b4148e39b11bedb2e1602c2` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:08.064961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc31cb1a-c984-40fd-a81e-cf3f366db8e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c527d333b4148e39b11bedb2e1602c2)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:08.069192 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc31cb1a-c984-40fd-a81e-cf3f366db8e3 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:08.130407 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-c117a016-0b8b-4b22-8f0e-3757248ece14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47d8e698d8ff4ea287b6a9f424c5f16e'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:08.148731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-c117a016-0b8b-4b22-8f0e-3757248ece14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47d8e698d8ff4ea287b6a9f424c5f16e'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:08.156393 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 917/1855] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:07:07 2022] DELETE /identity/v3/users/47d8e698d8ff4ea287b6a9f424c5f16e => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:08.157448 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117a016-0b8b-4b22-8f0e-3757248ece14 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:08.158884 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-c117a016-0b8b-4b22-8f0e-3757248ece14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:08.240547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fc31cb1a-c984-40fd-a81e-cf3f366db8e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c527d333b4148e39b11bedb2e1602c2', 'request_id': 'req-fc31cb1a-c984-40fd-a81e-cf3f366db8e3'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:08.249713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fc31cb1a-c984-40fd-a81e-cf3f366db8e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c527d333b4148e39b11bedb2e1602c2', 'request_id': 'req-fc31cb1a-c984-40fd-a81e-cf3f366db8e3'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:08.269658 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c117a016-0b8b-4b22-8f0e-3757248ece14 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:08.345283 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda89a92-2f3f-492f-af89-e01b5db51763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:08.346115 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda89a92-2f3f-492f-af89-e01b5db51763 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:08.346417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda89a92-2f3f-492f-af89-e01b5db51763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:08.346691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda89a92-2f3f-492f-af89-e01b5db51763 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:08.347391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda89a92-2f3f-492f-af89-e01b5db51763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:08.349040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda89a92-2f3f-492f-af89-e01b5db51763 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:08.354160 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fc31cb1a-c984-40fd-a81e-cf3f366db8e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c527d333b4148e39b11bedb2e1602c2', 'request_id': 'req-fc31cb1a-c984-40fd-a81e-cf3f366db8e3'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:08.361489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 918/1856] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:08.365449 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda89a92-2f3f-492f-af89-e01b5db51763 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:08.371165 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fc31cb1a-c984-40fd-a81e-cf3f366db8e3 admin admin] The token cache is being invalidate because project 5c527d333b4148e39b11bedb2e1602c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:07:08.371657 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fc31cb1a-c984-40fd-a81e-cf3f366db8e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:08.372990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fc31cb1a-c984-40fd-a81e-cf3f366db8e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:08.374709 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 939/1857] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:07:08 2022] DELETE /identity/v3/projects/5c527d333b4148e39b11bedb2e1602c2 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:08.378990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda89a92-2f3f-492f-af89-e01b5db51763 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:08.382370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ac5e5-b1b1-4b7a-9ae4-b335b5754c36 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:08.382888 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ac5e5-b1b1-4b7a-9ae4-b335b5754c36 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:08.383788 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ac5e5-b1b1-4b7a-9ae4-b335b5754c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:08.386803 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:08.410599 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ac25c-01f6-41e0-bba6-d74242b660b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:08.411927 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ac25c-01f6-41e0-bba6-d74242b660b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:08.412390 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ac25c-01f6-41e0-bba6-d74242b660b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:08.412795 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ac25c-01f6-41e0-bba6-d74242b660b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:08.413786 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ac25c-01f6-41e0-bba6-d74242b660b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:08.416867 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ac25c-01f6-41e0-bba6-d74242b660b3 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:08.417513 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-1b7ac5e5-b1b1-4b7a-9ae4-b335b5754c36 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:08.425592 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1b7ac5e5-b1b1-4b7a-9ae4-b335b5754c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:08.444329 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 919/1858] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:08.445716 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:08.445915 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ac25c-01f6-41e0-bba6-d74242b660b3 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:08.449331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-b89ac25c-01f6-41e0-bba6-d74242b660b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:08.457563 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 940/1859] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:08.465477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb438d7-c46e-454a-a9fd-7eb3c7c8a9be None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:08.466324 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb438d7-c46e-454a-a9fd-7eb3c7c8a9be None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:08.466619 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb438d7-c46e-454a-a9fd-7eb3c7c8a9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:08.476225 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ac25c-01f6-41e0-bba6-d74242b660b3 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:08.482695 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5500f-0e5e-4d34-a784-cbe7e47835c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:08.483879 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5500f-0e5e-4d34-a784-cbe7e47835c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:08.484458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5500f-0e5e-4d34-a784-cbe7e47835c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:08.484955 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5500f-0e5e-4d34-a784-cbe7e47835c5 admin admin] PATH_INFO: `/v3/projects/84ede0416d9644a98b8670239ccd0b55/users/d42c551062e340228f893cdfad3a1625/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:08.490730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-1bb438d7-c46e-454a-a9fd-7eb3c7c8a9be None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:08.491981 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1bb438d7-c46e-454a-a9fd-7eb3c7c8a9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:08.505828 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa5500f-0e5e-4d34-a784-cbe7e47835c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84ede0416d9644a98b8670239ccd0b55, user_id=d42c551062e340228f893cdfad3a1625, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:08.509489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa5500f-0e5e-4d34-a784-cbe7e47835c5 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:08.513659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 941/1860] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:08.518022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb438d7-c46e-454a-a9fd-7eb3c7c8a9be None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:08.530191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb438d7-c46e-454a-a9fd-7eb3c7c8a9be None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:08.536381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01d2ea-8616-4ae2-bfcf-8f8bb13d3bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:08.537035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01d2ea-8616-4ae2-bfcf-8f8bb13d3bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:08.537317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01d2ea-8616-4ae2-bfcf-8f8bb13d3bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:08.537597 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c01d2ea-8616-4ae2-bfcf-8f8bb13d3bc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:08.538354 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01d2ea-8616-4ae2-bfcf-8f8bb13d3bc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:07:08.538664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01d2ea-8616-4ae2-bfcf-8f8bb13d3bc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:08.540373 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c01d2ea-8616-4ae2-bfcf-8f8bb13d3bc3 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:08.554522 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 942/1861] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:07:08.559848 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01d2ea-8616-4ae2-bfcf-8f8bb13d3bc3 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:08.572008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c01d2ea-8616-4ae2-bfcf-8f8bb13d3bc3 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:08.582667 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936cec14-c7a5-4068-b40d-bf0dfb326254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:08.582667 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cec14-c7a5-4068-b40d-bf0dfb326254 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:08.582667 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cec14-c7a5-4068-b40d-bf0dfb326254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:08.582667 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cec14-c7a5-4068-b40d-bf0dfb326254 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:08.583005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936cec14-c7a5-4068-b40d-bf0dfb326254 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:08.583431 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936cec14-c7a5-4068-b40d-bf0dfb326254 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:08.618368 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 920/1862] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:08 2022] PUT /identity/v3/projects/84ede0416d9644a98b8670239ccd0b55/users/d42c551062e340228f893cdfad3a1625/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:08.629993 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a127fa4-74bd-41aa-a198-6faf914a77be None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:08.630281 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a127fa4-74bd-41aa-a198-6faf914a77be None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:08.630563 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a127fa4-74bd-41aa-a198-6faf914a77be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:08.667403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-8a127fa4-74bd-41aa-a198-6faf914a77be None None] MFA Rules not processed for user `d42c551062e340228f893cdfad3a1625`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:08.668721 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-8a127fa4-74bd-41aa-a198-6faf914a77be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:08.699009 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 943/1863] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:07:08 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:07:08.700492 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936cec14-c7a5-4068-b40d-bf0dfb326254 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:08.703599 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-936cec14-c7a5-4068-b40d-bf0dfb326254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:08.755326 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 921/1864] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:07:08.759085 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a127fa4-74bd-41aa-a198-6faf914a77be None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:08.788194 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936cec14-c7a5-4068-b40d-bf0dfb326254 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:08.794406 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed19e7-9f5c-4b67-9f98-b211ab5baf8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:08.795088 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed19e7-9f5c-4b67-9f98-b211ab5baf8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:08.795374 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed19e7-9f5c-4b67-9f98-b211ab5baf8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:08.795651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed19e7-9f5c-4b67-9f98-b211ab5baf8d admin admin] PATH_INFO: `/v3/projects/399ae69cf0f8418cba7e9e38e66466fd` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:08.797941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfed19e7-9f5c-4b67-9f98-b211ab5baf8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=399ae69cf0f8418cba7e9e38e66466fd)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:08.801467 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfed19e7-9f5c-4b67-9f98-b211ab5baf8d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:08.854364 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a127fa4-74bd-41aa-a198-6faf914a77be None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:08.859640 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c93760-ef0a-4b1d-8f3e-b1b49322b43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:08.860445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c93760-ef0a-4b1d-8f3e-b1b49322b43e admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:08.860587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c93760-ef0a-4b1d-8f3e-b1b49322b43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:08.860850 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c93760-ef0a-4b1d-8f3e-b1b49322b43e admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:08.861679 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c93760-ef0a-4b1d-8f3e-b1b49322b43e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:08.864471 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c93760-ef0a-4b1d-8f3e-b1b49322b43e admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:08.956626 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-dfed19e7-9f5c-4b67-9f98-b211ab5baf8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '399ae69cf0f8418cba7e9e38e66466fd', 'request_id': 'req-dfed19e7-9f5c-4b67-9f98-b211ab5baf8d'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:08.975879 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-dfed19e7-9f5c-4b67-9f98-b211ab5baf8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '399ae69cf0f8418cba7e9e38e66466fd', 'request_id': 'req-dfed19e7-9f5c-4b67-9f98-b211ab5baf8d'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:09.000194 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 922/1865] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:07:08 2022] POST /identity/v3/users => generated 293 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:09.001474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c93760-ef0a-4b1d-8f3e-b1b49322b43e admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:09.018778 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c93760-ef0a-4b1d-8f3e-b1b49322b43e admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:09.084769 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-dfed19e7-9f5c-4b67-9f98-b211ab5baf8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '399ae69cf0f8418cba7e9e38e66466fd', 'request_id': 'req-dfed19e7-9f5c-4b67-9f98-b211ab5baf8d'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:09.093063 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28159b27-1191-4f9e-adfb-36e3fb29f187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:09.093738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28159b27-1191-4f9e-adfb-36e3fb29f187 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:09.094307 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28159b27-1191-4f9e-adfb-36e3fb29f187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:09.094542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28159b27-1191-4f9e-adfb-36e3fb29f187 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:09.095325 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28159b27-1191-4f9e-adfb-36e3fb29f187 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:09.098898 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28159b27-1191-4f9e-adfb-36e3fb29f187 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:09.117938 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-dfed19e7-9f5c-4b67-9f98-b211ab5baf8d admin admin] The token cache is being invalidate because project 399ae69cf0f8418cba7e9e38e66466fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:07:09.117938 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-dfed19e7-9f5c-4b67-9f98-b211ab5baf8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:09.117938 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-dfed19e7-9f5c-4b67-9f98-b211ab5baf8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:09.125575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 944/1866] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:07:08 2022] DELETE /identity/v3/projects/399ae69cf0f8418cba7e9e38e66466fd => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:09.126865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed19e7-9f5c-4b67-9f98-b211ab5baf8d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:09.139650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-dfed19e7-9f5c-4b67-9f98-b211ab5baf8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:09.165476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed19e7-9f5c-4b67-9f98-b211ab5baf8d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:09.184548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d3948-556c-45fa-acd1-74853703bbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:09.185212 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d3948-556c-45fa-acd1-74853703bbe0 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:09.185491 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d3948-556c-45fa-acd1-74853703bbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:09.185775 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d3948-556c-45fa-acd1-74853703bbe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:09.186522 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d3948-556c-45fa-acd1-74853703bbe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:09.188176 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d3948-556c-45fa-acd1-74853703bbe0 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:09.201975 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 945/1867] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:09.203251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d80d05-4c04-41ec-8992-779d14faa421 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:09.203545 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d80d05-4c04-41ec-8992-779d14faa421 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:09.203822 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d80d05-4c04-41ec-8992-779d14faa421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:09.232124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f8d80d05-4c04-41ec-8992-779d14faa421 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:09.232804 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 923/1868] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:07:09 2022] POST /identity/v3/projects => generated 385 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:09.234420 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28159b27-1191-4f9e-adfb-36e3fb29f187 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:09.234758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f8d80d05-4c04-41ec-8992-779d14faa421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:09.324829 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28159b27-1191-4f9e-adfb-36e3fb29f187 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:09.330814 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b9a39-9c55-4ce6-abae-b244327f8c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:09.331446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b9a39-9c55-4ce6-abae-b244327f8c96 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:09.335951 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b9a39-9c55-4ce6-abae-b244327f8c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:09.335951 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b9a39-9c55-4ce6-abae-b244327f8c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:09.335951 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4b9a39-9c55-4ce6-abae-b244327f8c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:09.335951 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4b9a39-9c55-4ce6-abae-b244327f8c96 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:09.341466 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 946/1869] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:09.342482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d80d05-4c04-41ec-8992-779d14faa421 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:09.343856 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f8d80d05-4c04-41ec-8992-779d14faa421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:09.345870 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 924/1870] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:09.349353 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd448744-7fac-4bad-8aae-7ee115e678c8 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:09.349650 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd448744-7fac-4bad-8aae-7ee115e678c8 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:09.350031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd448744-7fac-4bad-8aae-7ee115e678c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:09.357416 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d80d05-4c04-41ec-8992-779d14faa421 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:09.362291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae333ba3-6ce3-4dd1-b774-1b7e334c39be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:09.362921 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae333ba3-6ce3-4dd1-b774-1b7e334c39be admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:09.363268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae333ba3-6ce3-4dd1-b774-1b7e334c39be admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:09.363551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae333ba3-6ce3-4dd1-b774-1b7e334c39be admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:09.368494 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae333ba3-6ce3-4dd1-b774-1b7e334c39be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:09.368494 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae333ba3-6ce3-4dd1-b774-1b7e334c39be admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:09.371822 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-bd448744-7fac-4bad-8aae-7ee115e678c8 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:09.373048 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-bd448744-7fac-4bad-8aae-7ee115e678c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:09.381095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 925/1871] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:09.381902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd448744-7fac-4bad-8aae-7ee115e678c8 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:09.392538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd448744-7fac-4bad-8aae-7ee115e678c8 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:09.397540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2582250-4424-4e50-9e7b-bdfccd432f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:09.398350 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2582250-4424-4e50-9e7b-bdfccd432f5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:09.398684 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2582250-4424-4e50-9e7b-bdfccd432f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:09.398991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2582250-4424-4e50-9e7b-bdfccd432f5f admin admin] PATH_INFO: `/v3/projects/2c5fd00315914450a1e0d3587459a904/users/113022328c224a26b3dc1a9d6b98b459/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:09.420116 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2582250-4424-4e50-9e7b-bdfccd432f5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c5fd00315914450a1e0d3587459a904, user_id=113022328c224a26b3dc1a9d6b98b459, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:09.420116 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2582250-4424-4e50-9e7b-bdfccd432f5f admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:09.467996 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 947/1872] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:07:09 2022] POST /identity/v3/users => generated 299 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:09.468966 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae333ba3-6ce3-4dd1-b774-1b7e334c39be admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:09.479778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae333ba3-6ce3-4dd1-b774-1b7e334c39be admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:09.484934 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5e264-9f94-46bf-a847-ae08b64ac5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:09.485645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5e264-9f94-46bf-a847-ae08b64ac5db admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:09.485861 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5e264-9f94-46bf-a847-ae08b64ac5db admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:09.486179 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5e264-9f94-46bf-a847-ae08b64ac5db admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:09.486911 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd5e264-9f94-46bf-a847-ae08b64ac5db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:07:09.487225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd5e264-9f94-46bf-a847-ae08b64ac5db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:09.489005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd5e264-9f94-46bf-a847-ae08b64ac5db admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:09.500195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 926/1873] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:09 2022] PUT /identity/v3/projects/2c5fd00315914450a1e0d3587459a904/users/113022328c224a26b3dc1a9d6b98b459/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:09.501668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2582250-4424-4e50-9e7b-bdfccd432f5f admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:09.508198 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 948/1874] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:07:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:07:09.514988 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5e264-9f94-46bf-a847-ae08b64ac5db admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:09.529495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd5e264-9f94-46bf-a847-ae08b64ac5db admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:09.587467 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2582250-4424-4e50-9e7b-bdfccd432f5f admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:09.592583 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bea96d-4377-42dc-aaa5-0ad07061b5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:09.593191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c936f-5ef6-49b8-896a-79a44f68bd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:09.593341 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bea96d-4377-42dc-aaa5-0ad07061b5b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:09.593488 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bea96d-4377-42dc-aaa5-0ad07061b5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:09.593764 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bea96d-4377-42dc-aaa5-0ad07061b5b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:09.593853 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c936f-5ef6-49b8-896a-79a44f68bd7f admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:09.594164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c936f-5ef6-49b8-896a-79a44f68bd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:09.594451 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881c936f-5ef6-49b8-896a-79a44f68bd7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:09.594562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bea96d-4377-42dc-aaa5-0ad07061b5b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:09.595143 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881c936f-5ef6-49b8-896a-79a44f68bd7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:09.596125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bea96d-4377-42dc-aaa5-0ad07061b5b9 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:09.597222 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881c936f-5ef6-49b8-896a-79a44f68bd7f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:09.609425 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 927/1875] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:09.611336 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fc49e-6f74-46ec-8c1d-cd830c9c79ef None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:09.611709 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fc49e-6f74-46ec-8c1d-cd830c9c79ef None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:09.611910 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8fc49e-6f74-46ec-8c1d-cd830c9c79ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:09.645330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-ec8fc49e-6f74-46ec-8c1d-cd830c9c79ef None None] MFA Rules not processed for user `113022328c224a26b3dc1a9d6b98b459`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:09.646716 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-ec8fc49e-6f74-46ec-8c1d-cd830c9c79ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:09.701924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 949/1876] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:07:09 2022] POST /identity/v3/projects => generated 373 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:09.702788 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c936f-5ef6-49b8-896a-79a44f68bd7f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:09.734336 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 928/1877] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:09.735435 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8fc49e-6f74-46ec-8c1d-cd830c9c79ef None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:09.752270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8fc49e-6f74-46ec-8c1d-cd830c9c79ef None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:09.783032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881c936f-5ef6-49b8-896a-79a44f68bd7f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:09.788111 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77daf59-6534-4ecf-8a20-9be56acf6fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:09.788800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77daf59-6534-4ecf-8a20-9be56acf6fcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:09.789548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77daf59-6534-4ecf-8a20-9be56acf6fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:09.789548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77daf59-6534-4ecf-8a20-9be56acf6fcc admin admin] PATH_INFO: `/v3/projects/c8199238dbe641e9a4005770807e9d40/users/bc3f16a57f1443689547f39ebd2083f9/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:09.809397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77daf59-6534-4ecf-8a20-9be56acf6fcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8199238dbe641e9a4005770807e9d40, user_id=bc3f16a57f1443689547f39ebd2083f9, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:09.814465 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77daf59-6534-4ecf-8a20-9be56acf6fcc admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:09.839944 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d309580-6608-4c53-9e6e-55988eabc65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:09.840947 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d309580-6608-4c53-9e6e-55988eabc65f admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:09.841388 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d309580-6608-4c53-9e6e-55988eabc65f admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:09.841841 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d309580-6608-4c53-9e6e-55988eabc65f admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:09.842716 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d309580-6608-4c53-9e6e-55988eabc65f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:09.845498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d309580-6608-4c53-9e6e-55988eabc65f admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:09.913105 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 950/1878] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:09 2022] PUT /identity/v3/projects/c8199238dbe641e9a4005770807e9d40/users/bc3f16a57f1443689547f39ebd2083f9/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:09.914113 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77daf59-6534-4ecf-8a20-9be56acf6fcc admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:09.915859 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b77daf59-6534-4ecf-8a20-9be56acf6fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:09.961217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 929/1879] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:07:09 2022] POST /identity/v3/users => generated 287 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:09.961217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d309580-6608-4c53-9e6e-55988eabc65f admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:09.976392 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d309580-6608-4c53-9e6e-55988eabc65f admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:09.983692 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77daf59-6534-4ecf-8a20-9be56acf6fcc admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:09.996832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58e901-21a7-4bee-a5b5-7602cfa5e5f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:09.996832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58e901-21a7-4bee-a5b5-7602cfa5e5f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:09.996832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58e901-21a7-4bee-a5b5-7602cfa5e5f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:09.996832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a58e901-21a7-4bee-a5b5-7602cfa5e5f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:10.079786 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a58e901-21a7-4bee-a5b5-7602cfa5e5f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:10.081662 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a58e901-21a7-4bee-a5b5-7602cfa5e5f3 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:10.088331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e080b-db5f-4f64-821e-7d54e97e33b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:10.089234 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e080b-db5f-4f64-821e-7d54e97e33b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:10.089732 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e080b-db5f-4f64-821e-7d54e97e33b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:10.090104 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e080b-db5f-4f64-821e-7d54e97e33b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:10.090902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e080b-db5f-4f64-821e-7d54e97e33b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:10.094195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e080b-db5f-4f64-821e-7d54e97e33b6 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:10.097855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 951/1880] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:07:10.098761 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58e901-21a7-4bee-a5b5-7602cfa5e5f3 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:10.113983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a58e901-21a7-4bee-a5b5-7602cfa5e5f3 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:10.118667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 930/1881] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:09 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:10.128129 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa447b7-f4d2-476c-a983-07630aecfd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:10.128129 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa447b7-f4d2-476c-a983-07630aecfd9d admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:10.128763 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e080b-db5f-4f64-821e-7d54e97e33b6 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:10.128896 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa447b7-f4d2-476c-a983-07630aecfd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:10.128896 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa447b7-f4d2-476c-a983-07630aecfd9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:10.128896 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa447b7-f4d2-476c-a983-07630aecfd9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:10.128896 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa447b7-f4d2-476c-a983-07630aecfd9d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:10.140170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e080b-db5f-4f64-821e-7d54e97e33b6 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:10.143944 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 952/1882] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:10.145067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa447b7-f4d2-476c-a983-07630aecfd9d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:10.149582 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df8ae5-ef98-494c-a737-a639b7906498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:10.151458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df8ae5-ef98-494c-a737-a639b7906498 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:10.152342 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df8ae5-ef98-494c-a737-a639b7906498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:10.153061 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df8ae5-ef98-494c-a737-a639b7906498 admin admin] PATH_INFO: `/v3/projects/c8199238dbe641e9a4005770807e9d40/users/bc3f16a57f1443689547f39ebd2083f9/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:10.156752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa447b7-f4d2-476c-a983-07630aecfd9d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:10.157711 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df8ae5-ef98-494c-a737-a639b7906498 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8199238dbe641e9a4005770807e9d40, user_id=bc3f16a57f1443689547f39ebd2083f9, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:10.161456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df8ae5-ef98-494c-a737-a639b7906498 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:10.163617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a22f59-62e3-414f-b1ab-a8010ccf30c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:10.164508 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a22f59-62e3-414f-b1ab-a8010ccf30c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:10.164906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a22f59-62e3-414f-b1ab-a8010ccf30c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:10.165359 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a22f59-62e3-414f-b1ab-a8010ccf30c9 admin admin] PATH_INFO: `/v3/users/113022328c224a26b3dc1a9d6b98b459` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:10.168886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a22f59-62e3-414f-b1ab-a8010ccf30c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=113022328c224a26b3dc1a9d6b98b459)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:10.171698 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a22f59-62e3-414f-b1ab-a8010ccf30c9 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:10.178602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 931/1883] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:10 2022] PUT /identity/v3/projects/c8199238dbe641e9a4005770807e9d40/users/bc3f16a57f1443689547f39ebd2083f9/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:07:10.179839 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df8ae5-ef98-494c-a737-a639b7906498 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:10.274139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df8ae5-ef98-494c-a737-a639b7906498 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:10.280828 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9036ad-414f-4f10-b564-60afcf85c088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:10.281689 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9036ad-414f-4f10-b564-60afcf85c088 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:10.282005 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9036ad-414f-4f10-b564-60afcf85c088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:10.282285 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9036ad-414f-4f10-b564-60afcf85c088 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:10.282984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9036ad-414f-4f10-b564-60afcf85c088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:10.284600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9036ad-414f-4f10-b564-60afcf85c088 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:10.296222 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 932/1884] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:10.297719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94092f1-1acf-4a67-ae5e-137ebca6f595 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:10.298141 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94092f1-1acf-4a67-ae5e-137ebca6f595 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:10.298265 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94092f1-1acf-4a67-ae5e-137ebca6f595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:10.308473 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:10.332599 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-a94092f1-1acf-4a67-ae5e-137ebca6f595 None None] MFA Rules not processed for user `bc3f16a57f1443689547f39ebd2083f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:10.334352 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-a94092f1-1acf-4a67-ae5e-137ebca6f595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:10.428579 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:10.444658 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 933/1885] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:10 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:10.445527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94092f1-1acf-4a67-ae5e-137ebca6f595 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:10.455117 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c6a22f59-62e3-414f-b1ab-a8010ccf30c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '113022328c224a26b3dc1a9d6b98b459'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:10.461656 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94092f1-1acf-4a67-ae5e-137ebca6f595 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:10.466001 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c6a22f59-62e3-414f-b1ab-a8010ccf30c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '113022328c224a26b3dc1a9d6b98b459'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:10.466861 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c6a22f59-62e3-414f-b1ab-a8010ccf30c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '113022328c224a26b3dc1a9d6b98b459'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:10.467212 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abebdbe-04f7-46aa-ad88-e73eca043caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:10.467920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abebdbe-04f7-46aa-ad88-e73eca043caf admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:10.468199 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abebdbe-04f7-46aa-ad88-e73eca043caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:10.468481 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abebdbe-04f7-46aa-ad88-e73eca043caf admin admin] PATH_INFO: `/v3/projects/c096035216b845379ee49e8c0c939a31/users/f9a3fb9d02134de3adc1e11f6933a601/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:10.487930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abebdbe-04f7-46aa-ad88-e73eca043caf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c096035216b845379ee49e8c0c939a31, user_id=f9a3fb9d02134de3adc1e11f6933a601, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:10.489414 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c6a22f59-62e3-414f-b1ab-a8010ccf30c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '113022328c224a26b3dc1a9d6b98b459'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:10.490668 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c6a22f59-62e3-414f-b1ab-a8010ccf30c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '113022328c224a26b3dc1a9d6b98b459'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:10.490936 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abebdbe-04f7-46aa-ad88-e73eca043caf admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:10.583499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 934/1886] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:10 2022] PUT /identity/v3/projects/c096035216b845379ee49e8c0c939a31/users/f9a3fb9d02134de3adc1e11f6933a601/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:10.584581 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abebdbe-04f7-46aa-ad88-e73eca043caf admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:10.587221 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-7abebdbe-04f7-46aa-ad88-e73eca043caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:10.598778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 953/1887] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:07:10 2022] DELETE /identity/v3/users/113022328c224a26b3dc1a9d6b98b459 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:10.600484 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cae4e-3eb8-485d-9ccc-c233c27fe179 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:10.600872 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cae4e-3eb8-485d-9ccc-c233c27fe179 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:10.601251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230cae4e-3eb8-485d-9ccc-c233c27fe179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:10.637950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-230cae4e-3eb8-485d-9ccc-c233c27fe179 None None] MFA Rules not processed for user `f9a3fb9d02134de3adc1e11f6933a601`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:10.639183 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-230cae4e-3eb8-485d-9ccc-c233c27fe179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:10.682760 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abebdbe-04f7-46aa-ad88-e73eca043caf admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:10.736190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 954/1888] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:10.736720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9143bd-7cf1-4caa-9692-cc68c24c5556 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:10.737354 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9143bd-7cf1-4caa-9692-cc68c24c5556 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:10.737490 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9312ef-c7ba-435c-a304-af6369a2bffb None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:10.737646 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9312ef-c7ba-435c-a304-af6369a2bffb None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:10.737804 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9143bd-7cf1-4caa-9692-cc68c24c5556 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:10.737981 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9312ef-c7ba-435c-a304-af6369a2bffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:10.738120 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9143bd-7cf1-4caa-9692-cc68c24c5556 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:10.743090 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-0e9143bd-7cf1-4caa-9692-cc68c24c5556 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:10.760938 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-aa9312ef-c7ba-435c-a304-af6369a2bffb None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:10.765329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-aa9312ef-c7ba-435c-a304-af6369a2bffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:10.823180 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9143bd-7cf1-4caa-9692-cc68c24c5556 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:10.824642 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9143bd-7cf1-4caa-9692-cc68c24c5556 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:10.836214 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 935/1889] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:07:10.839417 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 955/1890] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:10.850052 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9143bd-7cf1-4caa-9692-cc68c24c5556 service cinder] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:10.852033 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-0e9143bd-7cf1-4caa-9692-cc68c24c5556 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:10.863939 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9312ef-c7ba-435c-a304-af6369a2bffb None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:10.865656 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-aa9312ef-c7ba-435c-a304-af6369a2bffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:10.882910 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9312ef-c7ba-435c-a304-af6369a2bffb None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:10.887285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff6251-5d2f-4317-84c0-9857a6d91e26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:10.888077 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff6251-5d2f-4317-84c0-9857a6d91e26 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:10.888474 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff6251-5d2f-4317-84c0-9857a6d91e26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:10.888875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff6251-5d2f-4317-84c0-9857a6d91e26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:10.904881 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff6251-5d2f-4317-84c0-9857a6d91e26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:10.906624 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff6251-5d2f-4317-84c0-9857a6d91e26 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:10.919158 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 956/1891] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:07:10.920532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff6251-5d2f-4317-84c0-9857a6d91e26 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:10.921935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b8ff6251-5d2f-4317-84c0-9857a6d91e26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:10.939024 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9143bd-7cf1-4caa-9692-cc68c24c5556 service cinder] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:10.943923 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff6251-5d2f-4317-84c0-9857a6d91e26 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:10.949540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279da2e-f007-43fa-85fe-4cc87ad2252a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:10.950392 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279da2e-f007-43fa-85fe-4cc87ad2252a service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:10.950804 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279da2e-f007-43fa-85fe-4cc87ad2252a service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:10.951213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279da2e-f007-43fa-85fe-4cc87ad2252a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:10.951503 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec9915-4d06-4734-8989-a0be129a9878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:10.952330 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec9915-4d06-4734-8989-a0be129a9878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:10.952714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec9915-4d06-4734-8989-a0be129a9878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:10.952939 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec9915-4d06-4734-8989-a0be129a9878 admin admin] PATH_INFO: `/v3/users/d42c551062e340228f893cdfad3a1625` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:10.954212 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec9915-4d06-4734-8989-a0be129a9878 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d42c551062e340228f893cdfad3a1625)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:10.956580 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec9915-4d06-4734-8989-a0be129a9878 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:10.963902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279da2e-f007-43fa-85fe-4cc87ad2252a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:10.967210 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279da2e-f007-43fa-85fe-4cc87ad2252a service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:10.983100 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 936/1892] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:07:10.984801 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279da2e-f007-43fa-85fe-4cc87ad2252a service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:10.987194 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-7279da2e-f007-43fa-85fe-4cc87ad2252a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:11.003614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279da2e-f007-43fa-85fe-4cc87ad2252a service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:11.009317 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277634ee-5f2d-4d61-a41f-152a8e9a470f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:11.010030 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277634ee-5f2d-4d61-a41f-152a8e9a470f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:11.010718 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277634ee-5f2d-4d61-a41f-152a8e9a470f admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:11.011006 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277634ee-5f2d-4d61-a41f-152a8e9a470f admin admin] PATH_INFO: `/v3/users/f9a3fb9d02134de3adc1e11f6933a601` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:11.012667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277634ee-5f2d-4d61-a41f-152a8e9a470f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9a3fb9d02134de3adc1e11f6933a601)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:11.015285 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277634ee-5f2d-4d61-a41f-152a8e9a470f admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:11.194501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5eec9915-4d06-4734-8989-a0be129a9878 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd42c551062e340228f893cdfad3a1625'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:11.218104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5eec9915-4d06-4734-8989-a0be129a9878 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd42c551062e340228f893cdfad3a1625'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:11.219513 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5eec9915-4d06-4734-8989-a0be129a9878 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd42c551062e340228f893cdfad3a1625'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:11.245718 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5eec9915-4d06-4734-8989-a0be129a9878 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd42c551062e340228f893cdfad3a1625'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:11.246865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5eec9915-4d06-4734-8989-a0be129a9878 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd42c551062e340228f893cdfad3a1625'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:11.293432 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-277634ee-5f2d-4d61-a41f-152a8e9a470f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9a3fb9d02134de3adc1e11f6933a601'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:11.310878 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-277634ee-5f2d-4d61-a41f-152a8e9a470f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9a3fb9d02134de3adc1e11f6933a601'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:11.311492 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-277634ee-5f2d-4d61-a41f-152a8e9a470f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9a3fb9d02134de3adc1e11f6933a601'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:11.366937 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 957/1893] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:07:10 2022] DELETE /identity/v3/users/d42c551062e340228f893cdfad3a1625 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:11.376303 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df006ea-0b7f-42b0-8132-7359d1fcaec0 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:11.376303 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df006ea-0b7f-42b0-8132-7359d1fcaec0 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:11.376303 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df006ea-0b7f-42b0-8132-7359d1fcaec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:11.399744 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:11.410570 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-4df006ea-0b7f-42b0-8132-7359d1fcaec0 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:11.412629 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4df006ea-0b7f-42b0-8132-7359d1fcaec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:11.447882 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-277634ee-5f2d-4d61-a41f-152a8e9a470f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9a3fb9d02134de3adc1e11f6933a601'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:11.461861 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-277634ee-5f2d-4d61-a41f-152a8e9a470f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9a3fb9d02134de3adc1e11f6933a601'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:11.469257 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 937/1894] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:07:10 2022] DELETE /identity/v3/users/f9a3fb9d02134de3adc1e11f6933a601 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:11.473534 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547bf75-3e25-47d8-ac5e-931ac689a335 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:11.473971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547bf75-3e25-47d8-ac5e-931ac689a335 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:11.474468 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547bf75-3e25-47d8-ac5e-931ac689a335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:11.486753 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 958/1895] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:07:11.497576 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df006ea-0b7f-42b0-8132-7359d1fcaec0 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:11.499768 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-6547bf75-3e25-47d8-ac5e-931ac689a335 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:11.501105 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-6547bf75-3e25-47d8-ac5e-931ac689a335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:11.501237 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4df006ea-0b7f-42b0-8132-7359d1fcaec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:11.600454 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 938/1896] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:11.601549 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547bf75-3e25-47d8-ac5e-931ac689a335 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:11.603441 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-6547bf75-3e25-47d8-ac5e-931ac689a335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:11.617973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547bf75-3e25-47d8-ac5e-931ac689a335 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:11.626627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c0be5-2598-4b0c-8e87-0441c72f1293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:11.626627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c0be5-2598-4b0c-8e87-0441c72f1293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:11.626627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c0be5-2598-4b0c-8e87-0441c72f1293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:11.626627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c0be5-2598-4b0c-8e87-0441c72f1293 admin admin] PATH_INFO: `/v3/projects/2c5fd00315914450a1e0d3587459a904` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:11.627921 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c0be5-2598-4b0c-8e87-0441c72f1293 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c5fd00315914450a1e0d3587459a904)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:11.629975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c0be5-2598-4b0c-8e87-0441c72f1293 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:11.651050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df006ea-0b7f-42b0-8132-7359d1fcaec0 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:11.678538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:11.715730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742e9b7-d554-4332-bdbc-c4eea43e96ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:11.716360 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742e9b7-d554-4332-bdbc-c4eea43e96ec service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:11.716650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742e9b7-d554-4332-bdbc-c4eea43e96ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:11.716924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742e9b7-d554-4332-bdbc-c4eea43e96ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:11.727784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2742e9b7-d554-4332-bdbc-c4eea43e96ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:11.728733 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2742e9b7-d554-4332-bdbc-c4eea43e96ec service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:11.741159 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 959/1897] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:07:11.741927 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742e9b7-d554-4332-bdbc-c4eea43e96ec service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:11.747933 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-4e4c0be5-2598-4b0c-8e87-0441c72f1293 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c5fd00315914450a1e0d3587459a904', 'request_id': 'req-4e4c0be5-2598-4b0c-8e87-0441c72f1293'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:11.750899 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2742e9b7-d554-4332-bdbc-c4eea43e96ec service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:11.754555 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-4e4c0be5-2598-4b0c-8e87-0441c72f1293 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c5fd00315914450a1e0d3587459a904', 'request_id': 'req-4e4c0be5-2598-4b0c-8e87-0441c72f1293'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:11.761327 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-4e4c0be5-2598-4b0c-8e87-0441c72f1293 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c5fd00315914450a1e0d3587459a904', 'request_id': 'req-4e4c0be5-2598-4b0c-8e87-0441c72f1293'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:11.807095 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a846be4e-3274-4009-ae7a-477370814875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:11.807729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846be4e-3274-4009-ae7a-477370814875 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:11.808013 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846be4e-3274-4009-ae7a-477370814875 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:11.808291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a846be4e-3274-4009-ae7a-477370814875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:11.844904 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-4e4c0be5-2598-4b0c-8e87-0441c72f1293 admin admin] The token cache is being invalidate because project 2c5fd00315914450a1e0d3587459a904 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:07:11.845281 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-4e4c0be5-2598-4b0c-8e87-0441c72f1293 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:11.846201 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-4e4c0be5-2598-4b0c-8e87-0441c72f1293 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:11.848030 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 939/1898] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:07:11 2022] DELETE /identity/v3/projects/2c5fd00315914450a1e0d3587459a904 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:11.852966 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479429d0-5a3b-448d-8fb8-f88376d5dbec None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:11.853263 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479429d0-5a3b-448d-8fb8-f88376d5dbec None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:11.853836 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479429d0-5a3b-448d-8fb8-f88376d5dbec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:11.873264 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a846be4e-3274-4009-ae7a-477370814875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:11.874851 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a846be4e-3274-4009-ae7a-477370814875 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:11.875960 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-479429d0-5a3b-448d-8fb8-f88376d5dbec None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:11.877326 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-479429d0-5a3b-448d-8fb8-f88376d5dbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:11.884806 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 960/1899] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:07:11.886762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 940/1900] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:07:11.895926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12416f64-778f-4631-8d59-e715d30d094a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:11.896395 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12416f64-778f-4631-8d59-e715d30d094a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:11.896583 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12416f64-778f-4631-8d59-e715d30d094a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:11.920640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-12416f64-778f-4631-8d59-e715d30d094a None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:11.928600 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-12416f64-778f-4631-8d59-e715d30d094a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:11.934613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 961/1901] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:11.936112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479429d0-5a3b-448d-8fb8-f88376d5dbec None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:11.948929 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479429d0-5a3b-448d-8fb8-f88376d5dbec None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:11.956129 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1615de15-3763-47e8-bc69-dcfcecfa47d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:11.956902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615de15-3763-47e8-bc69-dcfcecfa47d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:11.963479 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615de15-3763-47e8-bc69-dcfcecfa47d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:11.963479 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1615de15-3763-47e8-bc69-dcfcecfa47d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:11.963479 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1615de15-3763-47e8-bc69-dcfcecfa47d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:07:11.963479 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1615de15-3763-47e8-bc69-dcfcecfa47d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:11.963479 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1615de15-3763-47e8-bc69-dcfcecfa47d1 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:11.974523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 941/1902] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:07:11.979029 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12416f64-778f-4631-8d59-e715d30d094a None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:11.988757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12416f64-778f-4631-8d59-e715d30d094a None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:11.994065 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e2f4d-3959-4609-9026-29e2f848c0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:11.994730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e2f4d-3959-4609-9026-29e2f848c0ca admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:11.995008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e2f4d-3959-4609-9026-29e2f848c0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:11.995289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278e2f4d-3959-4609-9026-29e2f848c0ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:11.996079 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278e2f4d-3959-4609-9026-29e2f848c0ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:11.998130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278e2f4d-3959-4609-9026-29e2f848c0ca admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:12.113408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 962/1903] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:07:11 2022] POST /identity/v3/projects => generated 375 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:12.114852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1615de15-3763-47e8-bc69-dcfcecfa47d1 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:12.126270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1615de15-3763-47e8-bc69-dcfcecfa47d1 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:12.206573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc046c-148f-4775-823b-fb19190f647e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:12.207621 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc046c-148f-4775-823b-fb19190f647e admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:12.208035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc046c-148f-4775-823b-fb19190f647e admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:12.208423 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc046c-148f-4775-823b-fb19190f647e admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:12.209424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc046c-148f-4775-823b-fb19190f647e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:12.211503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc046c-148f-4775-823b-fb19190f647e admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:12.292569 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e2f4d-3959-4609-9026-29e2f848c0ca admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:12.292921 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-278e2f4d-3959-4609-9026-29e2f848c0ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:12.310592 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278e2f4d-3959-4609-9026-29e2f848c0ca admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:12.316759 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5e92e-b798-4511-8916-fbcce57e9ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:12.317929 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5e92e-b798-4511-8916-fbcce57e9ba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:12.317929 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5e92e-b798-4511-8916-fbcce57e9ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:12.324230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5e92e-b798-4511-8916-fbcce57e9ba8 admin admin] PATH_INFO: `/v3/projects/84ede0416d9644a98b8670239ccd0b55` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:12.324230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5e92e-b798-4511-8916-fbcce57e9ba8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84ede0416d9644a98b8670239ccd0b55)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:12.324230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5e92e-b798-4511-8916-fbcce57e9ba8 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:12.324889 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 942/1904] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:07:12 2022] POST /identity/v3/users => generated 288 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:12.327138 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc046c-148f-4775-823b-fb19190f647e admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:12.338243 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc046c-148f-4775-823b-fb19190f647e admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:12.343394 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bc46f-1e55-413d-95ca-393ecf5de888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:12.344005 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bc46f-1e55-413d-95ca-393ecf5de888 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:12.344310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bc46f-1e55-413d-95ca-393ecf5de888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:12.344662 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bc46f-1e55-413d-95ca-393ecf5de888 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:12.345354 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bc46f-1e55-413d-95ca-393ecf5de888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:12.346990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bc46f-1e55-413d-95ca-393ecf5de888 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:12.359436 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 943/1905] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:12.364063 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bc46f-1e55-413d-95ca-393ecf5de888 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:12.374646 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bc46f-1e55-413d-95ca-393ecf5de888 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:12.380578 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f597365-e1b0-466a-84c8-2b7e9501f74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:12.381198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f597365-e1b0-466a-84c8-2b7e9501f74f admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:12.381485 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f597365-e1b0-466a-84c8-2b7e9501f74f admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:12.381798 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f597365-e1b0-466a-84c8-2b7e9501f74f admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:12.382550 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f597365-e1b0-466a-84c8-2b7e9501f74f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:12.384132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f597365-e1b0-466a-84c8-2b7e9501f74f admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:12.394554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 944/1906] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:07:12.400800 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f597365-e1b0-466a-84c8-2b7e9501f74f admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:12.411521 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f597365-e1b0-466a-84c8-2b7e9501f74f admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:12.418364 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd54b6f-0daa-4f36-993e-a60c6417d27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:12.419115 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd54b6f-0daa-4f36-993e-a60c6417d27b admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:12.422173 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd54b6f-0daa-4f36-993e-a60c6417d27b admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:12.422173 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd54b6f-0daa-4f36-993e-a60c6417d27b admin admin] PATH_INFO: `/v3/projects/dfd04ed2a8a845e7910bf7d24df1faa2/users/24ea439669ed4b5c92e7e9368aaace8b/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:12.435312 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd54b6f-0daa-4f36-993e-a60c6417d27b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfd04ed2a8a845e7910bf7d24df1faa2, user_id=24ea439669ed4b5c92e7e9368aaace8b, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:12.438273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd54b6f-0daa-4f36-993e-a60c6417d27b admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:12.531924 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 945/1907] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:12 2022] PUT /identity/v3/projects/dfd04ed2a8a845e7910bf7d24df1faa2/users/24ea439669ed4b5c92e7e9368aaace8b/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:07:12.533110 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd54b6f-0daa-4f36-993e-a60c6417d27b admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:12.534738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-1bd54b6f-0daa-4f36-993e-a60c6417d27b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:12.556376 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-75a5e92e-b798-4511-8916-fbcce57e9ba8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84ede0416d9644a98b8670239ccd0b55', 'request_id': 'req-75a5e92e-b798-4511-8916-fbcce57e9ba8'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:12.563703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-75a5e92e-b798-4511-8916-fbcce57e9ba8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84ede0416d9644a98b8670239ccd0b55', 'request_id': 'req-75a5e92e-b798-4511-8916-fbcce57e9ba8'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:12.606578 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd54b6f-0daa-4f36-993e-a60c6417d27b admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:12.633036 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-75a5e92e-b798-4511-8916-fbcce57e9ba8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84ede0416d9644a98b8670239ccd0b55', 'request_id': 'req-75a5e92e-b798-4511-8916-fbcce57e9ba8'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:12.650842 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-75a5e92e-b798-4511-8916-fbcce57e9ba8 admin admin] The token cache is being invalidate because project 84ede0416d9644a98b8670239ccd0b55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:07:12.651209 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-75a5e92e-b798-4511-8916-fbcce57e9ba8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:12.652077 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-75a5e92e-b798-4511-8916-fbcce57e9ba8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:12.653953 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 963/1908] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:07:12 2022] DELETE /identity/v3/projects/84ede0416d9644a98b8670239ccd0b55 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:07:12.655259 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aab9b1-a646-4700-b452-9218b6d0d3c9 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:12.655775 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aab9b1-a646-4700-b452-9218b6d0d3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:12.656130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aab9b1-a646-4700-b452-9218b6d0d3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:12.685191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892cf21-cbcc-43bb-a56b-38dc82a7571d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:12.686997 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892cf21-cbcc-43bb-a56b-38dc82a7571d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:12.686997 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892cf21-cbcc-43bb-a56b-38dc82a7571d admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:12.686997 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b892cf21-cbcc-43bb-a56b-38dc82a7571d admin admin] PATH_INFO: `/v3/projects/c096035216b845379ee49e8c0c939a31` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:12.689046 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b892cf21-cbcc-43bb-a56b-38dc82a7571d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c096035216b845379ee49e8c0c939a31)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:12.692555 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b892cf21-cbcc-43bb-a56b-38dc82a7571d admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:12.699648 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-25aab9b1-a646-4700-b452-9218b6d0d3c9 None None] MFA Rules not processed for user `24ea439669ed4b5c92e7e9368aaace8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:12.701675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-25aab9b1-a646-4700-b452-9218b6d0d3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:12.747970 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:12.795243 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 964/1909] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:12.795243 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aab9b1-a646-4700-b452-9218b6d0d3c9 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:12.795243 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-25aab9b1-a646-4700-b452-9218b6d0d3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:12.807501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aab9b1-a646-4700-b452-9218b6d0d3c9 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:12.816824 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4da8a2-72f1-4a2e-a924-bb40abd95934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:12.817495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4da8a2-72f1-4a2e-a924-bb40abd95934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:12.817775 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4da8a2-72f1-4a2e-a924-bb40abd95934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:12.818298 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4da8a2-72f1-4a2e-a924-bb40abd95934 admin admin] PATH_INFO: `/v3/users/bc3f16a57f1443689547f39ebd2083f9` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:12.820727 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4da8a2-72f1-4a2e-a924-bb40abd95934 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc3f16a57f1443689547f39ebd2083f9)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:12.823564 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4da8a2-72f1-4a2e-a924-bb40abd95934 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:12.866542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b892cf21-cbcc-43bb-a56b-38dc82a7571d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c096035216b845379ee49e8c0c939a31', 'request_id': 'req-b892cf21-cbcc-43bb-a56b-38dc82a7571d'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:12.873335 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b892cf21-cbcc-43bb-a56b-38dc82a7571d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c096035216b845379ee49e8c0c939a31', 'request_id': 'req-b892cf21-cbcc-43bb-a56b-38dc82a7571d'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:12.880028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b892cf21-cbcc-43bb-a56b-38dc82a7571d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c096035216b845379ee49e8c0c939a31', 'request_id': 'req-b892cf21-cbcc-43bb-a56b-38dc82a7571d'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:12.967494 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b892cf21-cbcc-43bb-a56b-38dc82a7571d admin admin] The token cache is being invalidate because project c096035216b845379ee49e8c0c939a31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:07:12.967960 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b892cf21-cbcc-43bb-a56b-38dc82a7571d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:12.969019 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b892cf21-cbcc-43bb-a56b-38dc82a7571d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:12.978247 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 946/1910] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:07:12 2022] DELETE /identity/v3/projects/c096035216b845379ee49e8c0c939a31 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:12.979206 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892cf21-cbcc-43bb-a56b-38dc82a7571d admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:12.980943 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-b892cf21-cbcc-43bb-a56b-38dc82a7571d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:13.051741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b892cf21-cbcc-43bb-a56b-38dc82a7571d admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:13.058204 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63ff93-e26d-40b0-b626-85bea5d0b5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:13.058906 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63ff93-e26d-40b0-b626-85bea5d0b5cd admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:13.059265 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63ff93-e26d-40b0-b626-85bea5d0b5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:13.059642 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63ff93-e26d-40b0-b626-85bea5d0b5cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:13.060838 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63ff93-e26d-40b0-b626-85bea5d0b5cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:13.060958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1f4da8a2-72f1-4a2e-a924-bb40abd95934 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc3f16a57f1443689547f39ebd2083f9'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:13.064229 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63ff93-e26d-40b0-b626-85bea5d0b5cd admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:13.068308 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1f4da8a2-72f1-4a2e-a924-bb40abd95934 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc3f16a57f1443689547f39ebd2083f9'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:13.068989 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1f4da8a2-72f1-4a2e-a924-bb40abd95934 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc3f16a57f1443689547f39ebd2083f9'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:13.082895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1f4da8a2-72f1-4a2e-a924-bb40abd95934 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc3f16a57f1443689547f39ebd2083f9'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:13.083618 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1f4da8a2-72f1-4a2e-a924-bb40abd95934 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc3f16a57f1443689547f39ebd2083f9'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:13.152975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:13.193945 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 965/1911] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:07:12 2022] DELETE /identity/v3/users/bc3f16a57f1443689547f39ebd2083f9 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:13.198124 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 947/1912] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:07:12 2022] POST /identity/v3/projects => generated 375 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:13.198287 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b3d1e-5b68-4aec-822b-291c401d4575 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:13.198287 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b3d1e-5b68-4aec-822b-291c401d4575 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:13.198287 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b3d1e-5b68-4aec-822b-291c401d4575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:13.205527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63ff93-e26d-40b0-b626-85bea5d0b5cd admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:13.207422 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4e63ff93-e26d-40b0-b626-85bea5d0b5cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:13.228980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-024b3d1e-5b68-4aec-822b-291c401d4575 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:13.230260 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-024b3d1e-5b68-4aec-822b-291c401d4575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:13.296662 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63ff93-e26d-40b0-b626-85bea5d0b5cd admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:13.302529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8cfb6-84f7-4624-beaf-394f1c7fe1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:13.303272 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8cfb6-84f7-4624-beaf-394f1c7fe1aa admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:13.303556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8cfb6-84f7-4624-beaf-394f1c7fe1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:13.303836 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8cfb6-84f7-4624-beaf-394f1c7fe1aa admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:13.305120 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad8cfb6-84f7-4624-beaf-394f1c7fe1aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:13.307016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 966/1913] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:13.311264 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad8cfb6-84f7-4624-beaf-394f1c7fe1aa admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:13.316397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dff7a5-e2b1-47a6-905c-321f23dd6a87 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:13.316770 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dff7a5-e2b1-47a6-905c-321f23dd6a87 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:13.317121 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dff7a5-e2b1-47a6-905c-321f23dd6a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:13.342355 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-78dff7a5-e2b1-47a6-905c-321f23dd6a87 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:13.343984 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-78dff7a5-e2b1-47a6-905c-321f23dd6a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:13.353841 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 967/1914] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:07:13.358331 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dff7a5-e2b1-47a6-905c-321f23dd6a87 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:13.367944 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dff7a5-e2b1-47a6-905c-321f23dd6a87 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:13.374267 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555fa1e9-2cb6-4903-86dd-16c9086a583a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:13.374548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fa1e9-2cb6-4903-86dd-16c9086a583a admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:13.374747 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fa1e9-2cb6-4903-86dd-16c9086a583a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:13.375022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fa1e9-2cb6-4903-86dd-16c9086a583a admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:13.375735 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555fa1e9-2cb6-4903-86dd-16c9086a583a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:07:13.376038 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555fa1e9-2cb6-4903-86dd-16c9086a583a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:13.377649 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555fa1e9-2cb6-4903-86dd-16c9086a583a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:13.391053 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 968/1915] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:07:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:07:13.395648 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555fa1e9-2cb6-4903-86dd-16c9086a583a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:13.407297 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555fa1e9-2cb6-4903-86dd-16c9086a583a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:13.412644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef2c91-5618-4429-a84b-646982ae6e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:13.413412 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef2c91-5618-4429-a84b-646982ae6e5d admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:13.413692 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef2c91-5618-4429-a84b-646982ae6e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:13.414223 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ef2c91-5618-4429-a84b-646982ae6e5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:13.414912 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef2c91-5618-4429-a84b-646982ae6e5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:13.420105 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ef2c91-5618-4429-a84b-646982ae6e5d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:13.462971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 948/1916] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:07:13 2022] POST /identity/v3/users => generated 294 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:13.467902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8cfb6-84f7-4624-beaf-394f1c7fe1aa admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:13.479196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8cfb6-84f7-4624-beaf-394f1c7fe1aa admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:13.485390 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1c035-e772-4f2f-880d-1f61be73c7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:13.486071 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1c035-e772-4f2f-880d-1f61be73c7e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:13.486349 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1c035-e772-4f2f-880d-1f61be73c7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:13.486663 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1c035-e772-4f2f-880d-1f61be73c7e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:13.488039 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1c035-e772-4f2f-880d-1f61be73c7e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:13.489691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba1c035-e772-4f2f-880d-1f61be73c7e6 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:13.510160 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 949/1917] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:07:13.514350 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1c035-e772-4f2f-880d-1f61be73c7e6 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:13.524125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1c035-e772-4f2f-880d-1f61be73c7e6 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:13.530535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eec432-b9db-482c-87c1-6f51b2c67d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:13.531373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eec432-b9db-482c-87c1-6f51b2c67d08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:13.531763 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eec432-b9db-482c-87c1-6f51b2c67d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:13.532205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eec432-b9db-482c-87c1-6f51b2c67d08 admin admin] PATH_INFO: `/v3/projects/93a5c2fb5015413a8219480fee1dc4cb/users/7d7d85a3b9474c7f9a92b7b45a518986/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:13.538799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 969/1918] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:07:13 2022] POST /identity/v3/projects => generated 361 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:07:13.547330 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef2c91-5618-4429-a84b-646982ae6e5d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:13.553220 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eec432-b9db-482c-87c1-6f51b2c67d08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a5c2fb5015413a8219480fee1dc4cb, user_id=7d7d85a3b9474c7f9a92b7b45a518986, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:13.556251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eec432-b9db-482c-87c1-6f51b2c67d08 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:13.621784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ef2c91-5618-4429-a84b-646982ae6e5d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:13.628149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e40768-af54-409e-9057-a34d88f2a1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:13.629127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e40768-af54-409e-9057-a34d88f2a1da admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:13.629414 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e40768-af54-409e-9057-a34d88f2a1da admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:13.629696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e40768-af54-409e-9057-a34d88f2a1da admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:13.630816 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e40768-af54-409e-9057-a34d88f2a1da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:13.632405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e40768-af54-409e-9057-a34d88f2a1da admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:13.653611 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 950/1919] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:13 2022] PUT /identity/v3/projects/93a5c2fb5015413a8219480fee1dc4cb/users/7d7d85a3b9474c7f9a92b7b45a518986/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:07:13.658910 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eec432-b9db-482c-87c1-6f51b2c67d08 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:13.741378 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 970/1920] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:07:13 2022] POST /identity/v3/users => generated 281 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:13.751014 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e40768-af54-409e-9057-a34d88f2a1da admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:13.762798 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e40768-af54-409e-9057-a34d88f2a1da admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:13.764195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eec432-b9db-482c-87c1-6f51b2c67d08 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:13.774342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9fadc-3095-4c45-890c-5495f38b309d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:13.775247 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9fadc-3095-4c45-890c-5495f38b309d admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:13.776165 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9fadc-3095-4c45-890c-5495f38b309d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:13.776542 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b9fadc-3095-4c45-890c-5495f38b309d admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:13.777392 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b9fadc-3095-4c45-890c-5495f38b309d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:13.778869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377dd07c-4f9f-4e9e-aadd-af5ffe7a4b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:13.779376 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b9fadc-3095-4c45-890c-5495f38b309d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:13.780460 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377dd07c-4f9f-4e9e-aadd-af5ffe7a4b21 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:13.780460 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377dd07c-4f9f-4e9e-aadd-af5ffe7a4b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:13.780792 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377dd07c-4f9f-4e9e-aadd-af5ffe7a4b21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:13.782063 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377dd07c-4f9f-4e9e-aadd-af5ffe7a4b21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:13.784793 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377dd07c-4f9f-4e9e-aadd-af5ffe7a4b21 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:13.799030 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 951/1921] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:13.799237 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 971/1922] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:13.804441 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377dd07c-4f9f-4e9e-aadd-af5ffe7a4b21 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:13.805641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9fadc-3095-4c45-890c-5495f38b309d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:13.817507 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377dd07c-4f9f-4e9e-aadd-af5ffe7a4b21 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:13.824845 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b9fadc-3095-4c45-890c-5495f38b309d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:13.826706 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23ccfb-6c55-4d8b-a8c3-84c42b9de9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:13.827854 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23ccfb-6c55-4d8b-a8c3-84c42b9de9bd admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:13.828324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23ccfb-6c55-4d8b-a8c3-84c42b9de9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:13.828848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23ccfb-6c55-4d8b-a8c3-84c42b9de9bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:13.831978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23ccfb-6c55-4d8b-a8c3-84c42b9de9bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:13.832165 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d9411-b512-4fd8-a6ed-f3cc97f182a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:13.832165 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d9411-b512-4fd8-a6ed-f3cc97f182a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:13.832165 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d9411-b512-4fd8-a6ed-f3cc97f182a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:13.840960 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d9411-b512-4fd8-a6ed-f3cc97f182a3 admin admin] PATH_INFO: `/v3/projects/93a5c2fb5015413a8219480fee1dc4cb/users/7d7d85a3b9474c7f9a92b7b45a518986/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:13.841466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23ccfb-6c55-4d8b-a8c3-84c42b9de9bd admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:13.841633 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d9411-b512-4fd8-a6ed-f3cc97f182a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a5c2fb5015413a8219480fee1dc4cb, user_id=7d7d85a3b9474c7f9a92b7b45a518986, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:13.841633 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d9411-b512-4fd8-a6ed-f3cc97f182a3 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:13.870518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 972/1923] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:13 2022] PUT /identity/v3/projects/93a5c2fb5015413a8219480fee1dc4cb/users/7d7d85a3b9474c7f9a92b7b45a518986/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:13.870518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d9411-b512-4fd8-a6ed-f3cc97f182a3 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:13.873784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a98d9411-b512-4fd8-a6ed-f3cc97f182a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:13.876272 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 952/1924] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:13.878732 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31533c1-fc1c-44c7-8a20-06f1711e2816 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:13.879363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31533c1-fc1c-44c7-8a20-06f1711e2816 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:13.880086 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31533c1-fc1c-44c7-8a20-06f1711e2816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:13.933299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-f31533c1-fc1c-44c7-8a20-06f1711e2816 None None] MFA Rules not processed for user `7d7d85a3b9474c7f9a92b7b45a518986`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:13.941122 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f31533c1-fc1c-44c7-8a20-06f1711e2816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:13.978761 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d9411-b512-4fd8-a6ed-f3cc97f182a3 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:13.984505 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b70764-dee3-4443-b3e1-971133436676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:13.986046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b70764-dee3-4443-b3e1-971133436676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:13.986425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b70764-dee3-4443-b3e1-971133436676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:13.986718 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b70764-dee3-4443-b3e1-971133436676 admin admin] PATH_INFO: `/v3/projects/c8199238dbe641e9a4005770807e9d40` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:13.991431 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b70764-dee3-4443-b3e1-971133436676 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8199238dbe641e9a4005770807e9d40)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:14.000057 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b70764-dee3-4443-b3e1-971133436676 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:14.010565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:14.025003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:14.076816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 953/1925] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:13 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:14.081750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31533c1-fc1c-44c7-8a20-06f1711e2816 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:14.094588 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31533c1-fc1c-44c7-8a20-06f1711e2816 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:14.099555 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a698c-e059-4fea-813b-d508bbaf41de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:14.100343 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a698c-e059-4fea-813b-d508bbaf41de admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:14.100694 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a698c-e059-4fea-813b-d508bbaf41de admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:14.101148 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8a698c-e059-4fea-813b-d508bbaf41de admin admin] PATH_INFO: `/v3/projects/1f3f2c032ec14edfa939d24224b702f7/users/7b8d25aa3ff144faa9484fd58be41d26/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:14.134117 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8a698c-e059-4fea-813b-d508bbaf41de admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f3f2c032ec14edfa939d24224b702f7, user_id=7b8d25aa3ff144faa9484fd58be41d26, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:14.138230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8a698c-e059-4fea-813b-d508bbaf41de admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:14.140639 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e5b70764-dee3-4443-b3e1-971133436676 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8199238dbe641e9a4005770807e9d40', 'request_id': 'req-e5b70764-dee3-4443-b3e1-971133436676'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:14.148104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e5b70764-dee3-4443-b3e1-971133436676 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8199238dbe641e9a4005770807e9d40', 'request_id': 'req-e5b70764-dee3-4443-b3e1-971133436676'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:14.209708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e5b70764-dee3-4443-b3e1-971133436676 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8199238dbe641e9a4005770807e9d40', 'request_id': 'req-e5b70764-dee3-4443-b3e1-971133436676'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:14.212425 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 954/1926] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:14 2022] PUT /identity/v3/projects/1f3f2c032ec14edfa939d24224b702f7/users/7b8d25aa3ff144faa9484fd58be41d26/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:14.213688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a698c-e059-4fea-813b-d508bbaf41de admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:14.223219 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-bc8a698c-e059-4fea-813b-d508bbaf41de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:14.224786 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e5b70764-dee3-4443-b3e1-971133436676 admin admin] The token cache is being invalidate because project c8199238dbe641e9a4005770807e9d40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:07:14.226886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e5b70764-dee3-4443-b3e1-971133436676 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:14.226886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e5b70764-dee3-4443-b3e1-971133436676 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:14.230464 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 973/1927] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:07:13 2022] DELETE /identity/v3/projects/c8199238dbe641e9a4005770807e9d40 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:14.233121 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee54352-db6f-4627-a29d-d9bf386ed46f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:14.233494 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee54352-db6f-4627-a29d-d9bf386ed46f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:14.233947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee54352-db6f-4627-a29d-d9bf386ed46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:14.278568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-eee54352-db6f-4627-a29d-d9bf386ed46f None None] MFA Rules not processed for user `7b8d25aa3ff144faa9484fd58be41d26`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:14.279926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-eee54352-db6f-4627-a29d-d9bf386ed46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:14.302131 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8a698c-e059-4fea-813b-d508bbaf41de admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:14.306047 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119842c3-18b2-444a-afad-60269118eba6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:14.307159 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119842c3-18b2-444a-afad-60269118eba6 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:14.307442 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119842c3-18b2-444a-afad-60269118eba6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:14.307715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119842c3-18b2-444a-afad-60269118eba6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:14.309658 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-119842c3-18b2-444a-afad-60269118eba6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:14.350659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 974/1928] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:14.351924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb444b4-1c2c-4184-87e9-004c8d697f6c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:14.352197 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb444b4-1c2c-4184-87e9-004c8d697f6c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:14.352473 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb444b4-1c2c-4184-87e9-004c8d697f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:14.382572 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-dbb444b4-1c2c-4184-87e9-004c8d697f6c None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:14.382572 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-dbb444b4-1c2c-4184-87e9-004c8d697f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:14.395550 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119842c3-18b2-444a-afad-60269118eba6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:14.397009 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119842c3-18b2-444a-afad-60269118eba6 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:14.406096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 955/1929] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:07:14.406898 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119842c3-18b2-444a-afad-60269118eba6 service cinder] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:14.408674 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-119842c3-18b2-444a-afad-60269118eba6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:14.420174 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119842c3-18b2-444a-afad-60269118eba6 service cinder] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:14.424075 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319215e-cecc-4d61-86ca-922b89e2d124 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:14.424754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319215e-cecc-4d61-86ca-922b89e2d124 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:14.424984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319215e-cecc-4d61-86ca-922b89e2d124 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:14.425655 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319215e-cecc-4d61-86ca-922b89e2d124 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:14.434775 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319215e-cecc-4d61-86ca-922b89e2d124 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:14.436203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319215e-cecc-4d61-86ca-922b89e2d124 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:14.442020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 975/1930] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:14.445813 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 956/1931] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:07:14.449509 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1920436-44b4-4c9a-a594-0e8e5e499e05 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:14.449802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1920436-44b4-4c9a-a594-0e8e5e499e05 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:14.450157 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1920436-44b4-4c9a-a594-0e8e5e499e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:14.463325 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319215e-cecc-4d61-86ca-922b89e2d124 service cinder] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:14.467335 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e319215e-cecc-4d61-86ca-922b89e2d124 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:14.491161 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319215e-cecc-4d61-86ca-922b89e2d124 service cinder] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:14.497124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-b1920436-44b4-4c9a-a594-0e8e5e499e05 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:14.498743 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b1920436-44b4-4c9a-a594-0e8e5e499e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:14.500087 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198b045-4cf7-4c45-b3b4-80a13eefa27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:14.501851 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198b045-4cf7-4c45-b3b4-80a13eefa27c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:14.501851 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198b045-4cf7-4c45-b3b4-80a13eefa27c admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:14.502210 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198b045-4cf7-4c45-b3b4-80a13eefa27c admin admin] PATH_INFO: `/v3/users/7b8d25aa3ff144faa9484fd58be41d26` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:14.504001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3198b045-4cf7-4c45-b3b4-80a13eefa27c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b8d25aa3ff144faa9484fd58be41d26)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:14.508038 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3198b045-4cf7-4c45-b3b4-80a13eefa27c admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:14.508500 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 976/1932] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:14.514302 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1920436-44b4-4c9a-a594-0e8e5e499e05 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:14.526734 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1920436-44b4-4c9a-a594-0e8e5e499e05 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:14.532015 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163d44c-d8a7-4604-8bf4-1072c0b1a7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:14.532654 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163d44c-d8a7-4604-8bf4-1072c0b1a7c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:14.532936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163d44c-d8a7-4604-8bf4-1072c0b1a7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:14.533614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163d44c-d8a7-4604-8bf4-1072c0b1a7c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:14.534373 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163d44c-d8a7-4604-8bf4-1072c0b1a7c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:07:14.534681 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163d44c-d8a7-4604-8bf4-1072c0b1a7c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:14.536064 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:14.536480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163d44c-d8a7-4604-8bf4-1072c0b1a7c1 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:14.552715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 977/1933] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:07:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:07:14.558405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163d44c-d8a7-4604-8bf4-1072c0b1a7c1 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:14.565992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:14.572666 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163d44c-d8a7-4604-8bf4-1072c0b1a7c1 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:14.578097 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0331a-710f-4ac5-b16e-064f1c6cf453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:14.578783 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0331a-710f-4ac5-b16e-064f1c6cf453 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:14.579907 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0331a-710f-4ac5-b16e-064f1c6cf453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:14.579907 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e0331a-710f-4ac5-b16e-064f1c6cf453 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:14.581260 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e0331a-710f-4ac5-b16e-064f1c6cf453 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:14.582535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e0331a-710f-4ac5-b16e-064f1c6cf453 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:14.725911 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 978/1934] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:07:14 2022] POST /identity/v3/projects => generated 391 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:14.728588 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0331a-710f-4ac5-b16e-064f1c6cf453 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:14.757312 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-3198b045-4cf7-4c45-b3b4-80a13eefa27c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b8d25aa3ff144faa9484fd58be41d26'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:14.770049 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-3198b045-4cf7-4c45-b3b4-80a13eefa27c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b8d25aa3ff144faa9484fd58be41d26'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:14.771204 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-3198b045-4cf7-4c45-b3b4-80a13eefa27c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b8d25aa3ff144faa9484fd58be41d26'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:14.801351 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e0331a-710f-4ac5-b16e-064f1c6cf453 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:14.806281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc1a74-d87d-43e1-b1e8-89a7083401f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:14.807453 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc1a74-d87d-43e1-b1e8-89a7083401f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:14.807743 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc1a74-d87d-43e1-b1e8-89a7083401f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:14.808156 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc1a74-d87d-43e1-b1e8-89a7083401f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:14.808919 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc1a74-d87d-43e1-b1e8-89a7083401f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:14.811102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc1a74-d87d-43e1-b1e8-89a7083401f4 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:14.851110 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-3198b045-4cf7-4c45-b3b4-80a13eefa27c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b8d25aa3ff144faa9484fd58be41d26'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:14.874623 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-3198b045-4cf7-4c45-b3b4-80a13eefa27c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b8d25aa3ff144faa9484fd58be41d26'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:14.892386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 979/1935] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:07:14 2022] POST /identity/v3/users => generated 296 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:14.895373 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc1a74-d87d-43e1-b1e8-89a7083401f4 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:14.899028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 957/1936] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:07:14 2022] DELETE /identity/v3/users/7b8d25aa3ff144faa9484fd58be41d26 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:14.899167 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-90bc1a74-d87d-43e1-b1e8-89a7083401f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:14.902359 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69efed5-c481-4a24-b8c6-a578803f48b5 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:14.902852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69efed5-c481-4a24-b8c6-a578803f48b5 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:14.903379 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69efed5-c481-4a24-b8c6-a578803f48b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:14.917035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc1a74-d87d-43e1-b1e8-89a7083401f4 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:14.964807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-d69efed5-c481-4a24-b8c6-a578803f48b5 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:14.966914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-d69efed5-c481-4a24-b8c6-a578803f48b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:15.012251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c4cc1-eac3-451b-8dcf-c43e474ccb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:15.013615 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c4cc1-eac3-451b-8dcf-c43e474ccb9d admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:15.013615 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c4cc1-eac3-451b-8dcf-c43e474ccb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:15.014043 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c4cc1-eac3-451b-8dcf-c43e474ccb9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:15.015223 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673c4cc1-eac3-451b-8dcf-c43e474ccb9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:15.018090 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673c4cc1-eac3-451b-8dcf-c43e474ccb9d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:15.032484 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 980/1937] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:07:15.036915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c4cc1-eac3-451b-8dcf-c43e474ccb9d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:15.051942 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c4cc1-eac3-451b-8dcf-c43e474ccb9d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:15.058272 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89aa475-6931-4973-8686-f3c860981c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:15.059150 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89aa475-6931-4973-8686-f3c860981c4c admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:15.059430 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89aa475-6931-4973-8686-f3c860981c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:15.059777 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89aa475-6931-4973-8686-f3c860981c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:15.060607 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89aa475-6931-4973-8686-f3c860981c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:15.062672 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89aa475-6931-4973-8686-f3c860981c4c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:15.078730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 981/1938] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:15.079999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89aa475-6931-4973-8686-f3c860981c4c admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:15.092564 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89aa475-6931-4973-8686-f3c860981c4c admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:15.097724 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 958/1939] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:15.106785 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069b332-71c3-4393-a7be-0bd6ad627436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:15.107568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069b332-71c3-4393-a7be-0bd6ad627436 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:15.107946 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069b332-71c3-4393-a7be-0bd6ad627436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:15.108236 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069b332-71c3-4393-a7be-0bd6ad627436 admin admin] PATH_INFO: `/v3/projects/f7c6d9785c014fb880ceb507f0e86fe3/users/7ed9547368734943916f581193fa51ae/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:15.130055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69efed5-c481-4a24-b8c6-a578803f48b5 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:15.130055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-d69efed5-c481-4a24-b8c6-a578803f48b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:15.130378 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069b332-71c3-4393-a7be-0bd6ad627436 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7c6d9785c014fb880ceb507f0e86fe3, user_id=7ed9547368734943916f581193fa51ae, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:15.134109 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069b332-71c3-4393-a7be-0bd6ad627436 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:15.197974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69efed5-c481-4a24-b8c6-a578803f48b5 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:15.202005 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c2e79-d31d-4e71-9e59-779fb895a341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:15.202596 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c2e79-d31d-4e71-9e59-779fb895a341 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:15.202879 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c2e79-d31d-4e71-9e59-779fb895a341 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:15.203155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c2e79-d31d-4e71-9e59-779fb895a341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:15.213316 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c2e79-d31d-4e71-9e59-779fb895a341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:15.214957 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c2e79-d31d-4e71-9e59-779fb895a341 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:15.234850 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 959/1940] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:07:15.263833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 982/1941] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:15 2022] PUT /identity/v3/projects/f7c6d9785c014fb880ceb507f0e86fe3/users/7ed9547368734943916f581193fa51ae/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:15.273248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de6a73-96f0-4ba9-b0e0-922380549c40 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:15.273819 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de6a73-96f0-4ba9-b0e0-922380549c40 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:15.274342 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de6a73-96f0-4ba9-b0e0-922380549c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:15.325497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-c1de6a73-96f0-4ba9-b0e0-922380549c40 None None] MFA Rules not processed for user `7ed9547368734943916f581193fa51ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:15.326751 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-c1de6a73-96f0-4ba9-b0e0-922380549c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:15.420056 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 960/1942] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:15 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:15.424388 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069b332-71c3-4393-a7be-0bd6ad627436 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:15.484748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:15.499184 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069b332-71c3-4393-a7be-0bd6ad627436 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:15.504072 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414202f-f156-4f7c-a9de-434f79ea4776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:15.504705 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414202f-f156-4f7c-a9de-434f79ea4776 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:15.504971 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414202f-f156-4f7c-a9de-434f79ea4776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:15.505319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414202f-f156-4f7c-a9de-434f79ea4776 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:15.506034 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414202f-f156-4f7c-a9de-434f79ea4776 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:15.507942 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414202f-f156-4f7c-a9de-434f79ea4776 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:15.606027 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:15.617878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 983/1943] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:07:15 2022] POST /identity/v3/projects => generated 391 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:15.620365 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de6a73-96f0-4ba9-b0e0-922380549c40 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:15.636979 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de6a73-96f0-4ba9-b0e0-922380549c40 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:15.703285 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b863a-e9f5-488c-a78c-62ad0fd9ad05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:15.703988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b863a-e9f5-488c-a78c-62ad0fd9ad05 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:15.704416 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b863a-e9f5-488c-a78c-62ad0fd9ad05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:15.705004 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b863a-e9f5-488c-a78c-62ad0fd9ad05 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:15.705994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342b863a-e9f5-488c-a78c-62ad0fd9ad05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:15.707672 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342b863a-e9f5-488c-a78c-62ad0fd9ad05 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:15.798927 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 961/1944] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:07:15 2022] POST /identity/v3/users => generated 302 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:15.803042 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414202f-f156-4f7c-a9de-434f79ea4776 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:15.821739 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414202f-f156-4f7c-a9de-434f79ea4776 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:15.830264 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af14ad-f95a-4cce-9edf-e06dc596bee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:15.831266 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af14ad-f95a-4cce-9edf-e06dc596bee4 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:15.831783 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af14ad-f95a-4cce-9edf-e06dc596bee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:15.832281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af14ad-f95a-4cce-9edf-e06dc596bee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:15.833511 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af14ad-f95a-4cce-9edf-e06dc596bee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:15.836525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af14ad-f95a-4cce-9edf-e06dc596bee4 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:15.857421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 984/1945] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:07:15.857590 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b863a-e9f5-488c-a78c-62ad0fd9ad05 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:15.861272 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af14ad-f95a-4cce-9edf-e06dc596bee4 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:15.864221 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a4af14ad-f95a-4cce-9edf-e06dc596bee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:15.877439 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342b863a-e9f5-488c-a78c-62ad0fd9ad05 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:15.879480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af14ad-f95a-4cce-9edf-e06dc596bee4 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:15.885067 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea50458-d523-46be-a3d7-9123c59f8578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:15.886059 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d790451-5d9a-4401-8dce-756d42ec1eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:15.886190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea50458-d523-46be-a3d7-9123c59f8578 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:15.886190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea50458-d523-46be-a3d7-9123c59f8578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:15.886499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea50458-d523-46be-a3d7-9123c59f8578 admin admin] PATH_INFO: `/v3/projects/54a9ade34599442fb03bcd3215fc0121/users/7c13f09f637c4ccd87e0b5036a1602e4/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:15.890800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d790451-5d9a-4401-8dce-756d42ec1eae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:15.891097 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d790451-5d9a-4401-8dce-756d42ec1eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:15.891382 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d790451-5d9a-4401-8dce-756d42ec1eae admin admin] PATH_INFO: `/v3/projects/1f3f2c032ec14edfa939d24224b702f7` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:15.892910 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d790451-5d9a-4401-8dce-756d42ec1eae admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f3f2c032ec14edfa939d24224b702f7)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:15.901931 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d790451-5d9a-4401-8dce-756d42ec1eae admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:15.905379 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea50458-d523-46be-a3d7-9123c59f8578 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54a9ade34599442fb03bcd3215fc0121, user_id=7c13f09f637c4ccd87e0b5036a1602e4, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:15.908106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea50458-d523-46be-a3d7-9123c59f8578 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:16.016757 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 962/1946] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:15 2022] PUT /identity/v3/projects/54a9ade34599442fb03bcd3215fc0121/users/7c13f09f637c4ccd87e0b5036a1602e4/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:16.020808 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea50458-d523-46be-a3d7-9123c59f8578 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:16.046410 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8d790451-5d9a-4401-8dce-756d42ec1eae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f3f2c032ec14edfa939d24224b702f7', 'request_id': 'req-8d790451-5d9a-4401-8dce-756d42ec1eae'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:16.058172 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8d790451-5d9a-4401-8dce-756d42ec1eae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f3f2c032ec14edfa939d24224b702f7', 'request_id': 'req-8d790451-5d9a-4401-8dce-756d42ec1eae'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:16.086680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea50458-d523-46be-a3d7-9123c59f8578 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:16.136251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8d790451-5d9a-4401-8dce-756d42ec1eae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f3f2c032ec14edfa939d24224b702f7', 'request_id': 'req-8d790451-5d9a-4401-8dce-756d42ec1eae'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:16.148603 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d6679d-c7d8-42da-b1f8-94738aced387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:16.150875 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6679d-c7d8-42da-b1f8-94738aced387 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:16.150875 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6679d-c7d8-42da-b1f8-94738aced387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:16.150875 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d6679d-c7d8-42da-b1f8-94738aced387 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:16.152008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d6679d-c7d8-42da-b1f8-94738aced387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:16.154027 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8d790451-5d9a-4401-8dce-756d42ec1eae admin admin] The token cache is being invalidate because project 1f3f2c032ec14edfa939d24224b702f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:07:16.154256 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d6679d-c7d8-42da-b1f8-94738aced387 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:16.154366 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8d790451-5d9a-4401-8dce-756d42ec1eae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:16.155014 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8d790451-5d9a-4401-8dce-756d42ec1eae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:16.156602 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 985/1947] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:07:15 2022] DELETE /identity/v3/projects/1f3f2c032ec14edfa939d24224b702f7 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:16.163294 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3108c89-483e-402b-8b92-4b6446be6cc4 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:16.163649 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3108c89-483e-402b-8b92-4b6446be6cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:16.163958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3108c89-483e-402b-8b92-4b6446be6cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:16.170022 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 963/1948] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:07:16.171048 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d6679d-c7d8-42da-b1f8-94738aced387 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:16.175890 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-d8d6679d-c7d8-42da-b1f8-94738aced387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:16.186112 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f3108c89-483e-402b-8b92-4b6446be6cc4 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:16.187648 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f3108c89-483e-402b-8b92-4b6446be6cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:16.195598 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d6679d-c7d8-42da-b1f8-94738aced387 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:16.196852 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 986/1949] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:07:16.204116 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b85ea-7666-4fd2-af95-f78a256a5c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:16.204363 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5cf9b-dece-4f72-a9f5-a3eb697902f3 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:16.204509 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5cf9b-dece-4f72-a9f5-a3eb697902f3 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:16.205045 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b5cf9b-dece-4f72-a9f5-a3eb697902f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:16.205194 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b85ea-7666-4fd2-af95-f78a256a5c61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:16.205376 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b85ea-7666-4fd2-af95-f78a256a5c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:16.205680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b85ea-7666-4fd2-af95-f78a256a5c61 admin admin] PATH_INFO: `/v3/projects/54a9ade34599442fb03bcd3215fc0121/users/7c13f09f637c4ccd87e0b5036a1602e4/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:16.207748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b85ea-7666-4fd2-af95-f78a256a5c61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54a9ade34599442fb03bcd3215fc0121, user_id=7c13f09f637c4ccd87e0b5036a1602e4, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:16.210684 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24b85ea-7666-4fd2-af95-f78a256a5c61 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:16.225616 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 964/1950] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:16 2022] PUT /identity/v3/projects/54a9ade34599442fb03bcd3215fc0121/users/7c13f09f637c4ccd87e0b5036a1602e4/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:07:16.233125 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-88b5cf9b-dece-4f72-a9f5-a3eb697902f3 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:16.234430 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-88b5cf9b-dece-4f72-a9f5-a3eb697902f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:16.237845 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cae0e5-c017-4ada-9f5d-fe99930ff052 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:16.238497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cae0e5-c017-4ada-9f5d-fe99930ff052 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:16.238893 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cae0e5-c017-4ada-9f5d-fe99930ff052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:16.272588 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-59cae0e5-c017-4ada-9f5d-fe99930ff052 None None] MFA Rules not processed for user `7c13f09f637c4ccd87e0b5036a1602e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:16.274148 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-59cae0e5-c017-4ada-9f5d-fe99930ff052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:16.337472 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 987/1951] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:07:16.339358 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5cf9b-dece-4f72-a9f5-a3eb697902f3 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:16.351768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b5cf9b-dece-4f72-a9f5-a3eb697902f3 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:16.360651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79733c-a809-4038-b996-725fb79e6448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:16.361293 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79733c-a809-4038-b996-725fb79e6448 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:16.365984 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79733c-a809-4038-b996-725fb79e6448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:16.366473 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79733c-a809-4038-b996-725fb79e6448 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:16.367512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79733c-a809-4038-b996-725fb79e6448 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:07:16.373845 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79733c-a809-4038-b996-725fb79e6448 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:16.374249 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 965/1952] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:16 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:07:16.374403 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79733c-a809-4038-b996-725fb79e6448 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:16.384084 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cae0e5-c017-4ada-9f5d-fe99930ff052 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:16.386247 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-59cae0e5-c017-4ada-9f5d-fe99930ff052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:16.392010 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 988/1953] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:07:16.393044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79733c-a809-4038-b996-725fb79e6448 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:16.403923 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79733c-a809-4038-b996-725fb79e6448 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:16.408830 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417449f9-e8ea-4a0e-9456-e54f305417e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:16.409392 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417449f9-e8ea-4a0e-9456-e54f305417e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:16.409693 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417449f9-e8ea-4a0e-9456-e54f305417e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:16.409989 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417449f9-e8ea-4a0e-9456-e54f305417e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:16.410665 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417449f9-e8ea-4a0e-9456-e54f305417e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:16.412380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=81,keystone:INSERT=2,keystone:DELETE=17 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:16.412577 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417449f9-e8ea-4a0e-9456-e54f305417e5 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:16.438072 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:16.456259 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cae0e5-c017-4ada-9f5d-fe99930ff052 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:16.460073 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436eed36-8504-4e4e-90ee-e255ed6b3fdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:16.460710 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436eed36-8504-4e4e-90ee-e255ed6b3fdb service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:16.460992 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436eed36-8504-4e4e-90ee-e255ed6b3fdb service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:16.461266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436eed36-8504-4e4e-90ee-e255ed6b3fdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:16.479294 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436eed36-8504-4e4e-90ee-e255ed6b3fdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:16.481026 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436eed36-8504-4e4e-90ee-e255ed6b3fdb service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:16.498438 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 966/1954] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:16 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:07:16.538069 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 989/1955] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:07:16 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:16.543035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436eed36-8504-4e4e-90ee-e255ed6b3fdb service cinder] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:16.557495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417449f9-e8ea-4a0e-9456-e54f305417e5 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:16.559396 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-417449f9-e8ea-4a0e-9456-e54f305417e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:16.560881 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436eed36-8504-4e4e-90ee-e255ed6b3fdb service cinder] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:16.646165 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417449f9-e8ea-4a0e-9456-e54f305417e5 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:16.650299 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071b4f9-3bd5-4d6d-a5ad-7cd23fac8d75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:16.651200 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071b4f9-3bd5-4d6d-a5ad-7cd23fac8d75 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:16.651586 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071b4f9-3bd5-4d6d-a5ad-7cd23fac8d75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:16.651968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071b4f9-3bd5-4d6d-a5ad-7cd23fac8d75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:16.654038 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7071b4f9-3bd5-4d6d-a5ad-7cd23fac8d75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:16.673918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3859f4-3b84-4d05-90e1-71f088f43253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:16.673918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3859f4-3b84-4d05-90e1-71f088f43253 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:16.673918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3859f4-3b84-4d05-90e1-71f088f43253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:16.673918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3859f4-3b84-4d05-90e1-71f088f43253 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:16.674401 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3859f4-3b84-4d05-90e1-71f088f43253 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:16.674401 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3859f4-3b84-4d05-90e1-71f088f43253 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:16.727168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7071b4f9-3bd5-4d6d-a5ad-7cd23fac8d75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:16.729219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7071b4f9-3bd5-4d6d-a5ad-7cd23fac8d75 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:16.739043 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 990/1956] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:16 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:07:16.792212 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 967/1957] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:07:16 2022] POST /identity/v3/users => generated 293 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:16.793433 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071b4f9-3bd5-4d6d-a5ad-7cd23fac8d75 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:16.804397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071b4f9-3bd5-4d6d-a5ad-7cd23fac8d75 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:16.814102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abc7fa-b7db-4aee-a362-03c15fb81987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:16.814767 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abc7fa-b7db-4aee-a362-03c15fb81987 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:16.815012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abc7fa-b7db-4aee-a362-03c15fb81987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:16.815292 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abc7fa-b7db-4aee-a362-03c15fb81987 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:16.815989 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abc7fa-b7db-4aee-a362-03c15fb81987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:16.817597 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abc7fa-b7db-4aee-a362-03c15fb81987 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:16.828556 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 991/1958] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:07:16.832745 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3859f4-3b84-4d05-90e1-71f088f43253 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:16.844366 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3859f4-3b84-4d05-90e1-71f088f43253 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:16.849300 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f688ce-78bd-46f2-baf0-cbf58cea1fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:16.852807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f688ce-78bd-46f2-baf0-cbf58cea1fed admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:16.852807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f688ce-78bd-46f2-baf0-cbf58cea1fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:16.852807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f688ce-78bd-46f2-baf0-cbf58cea1fed admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:16.852807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f688ce-78bd-46f2-baf0-cbf58cea1fed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:16.853321 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f688ce-78bd-46f2-baf0-cbf58cea1fed admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:16.864982 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 968/1959] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:16.870856 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abc7fa-b7db-4aee-a362-03c15fb81987 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:16.890584 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abc7fa-b7db-4aee-a362-03c15fb81987 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:16.897387 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754a66e-4432-4deb-aa87-53616d949761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:16.897697 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754a66e-4432-4deb-aa87-53616d949761 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:16.898035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754a66e-4432-4deb-aa87-53616d949761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:16.898477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9754a66e-4432-4deb-aa87-53616d949761 admin admin] PATH_INFO: `/v3/projects/5132918a404e4b6ab13577698e319630/users/8b2daf470f45422a8c1bd5cf62efdd9e/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:16.919402 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9754a66e-4432-4deb-aa87-53616d949761 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5132918a404e4b6ab13577698e319630, user_id=8b2daf470f45422a8c1bd5cf62efdd9e, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:16.922775 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9754a66e-4432-4deb-aa87-53616d949761 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:17.014697 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 992/1960] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:16 2022] PUT /identity/v3/projects/5132918a404e4b6ab13577698e319630/users/8b2daf470f45422a8c1bd5cf62efdd9e/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:17.022055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7586aa-1f09-419b-9f8d-1eca2a47cb15 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:17.022544 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7586aa-1f09-419b-9f8d-1eca2a47cb15 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:17.023015 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7586aa-1f09-419b-9f8d-1eca2a47cb15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:17.046665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-4f7586aa-1f09-419b-9f8d-1eca2a47cb15 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:17.047872 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4f7586aa-1f09-419b-9f8d-1eca2a47cb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:17.115522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 969/1961] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:17.125335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754a66e-4432-4deb-aa87-53616d949761 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:17.126667 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9754a66e-4432-4deb-aa87-53616d949761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:17.164595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:17.199134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9754a66e-4432-4deb-aa87-53616d949761 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:17.205149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df15f1-5cf7-49ad-b59c-c41eb6d92c20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:17.205744 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df15f1-5cf7-49ad-b59c-c41eb6d92c20 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:17.206080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df15f1-5cf7-49ad-b59c-c41eb6d92c20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:17.206352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df15f1-5cf7-49ad-b59c-c41eb6d92c20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:17.218274 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df15f1-5cf7-49ad-b59c-c41eb6d92c20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:17.219896 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df15f1-5cf7-49ad-b59c-c41eb6d92c20 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:17.232446 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 993/1962] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:07:17.241923 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:18.111414 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:18.222517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:18.868290 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:18.942804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:19.371757 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:20.050157 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:20.366308 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:21.260039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:21.605972 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:22.762945 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7586aa-1f09-419b-9f8d-1eca2a47cb15 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:22.765232 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4f7586aa-1f09-419b-9f8d-1eca2a47cb15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:22.814845 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7586aa-1f09-419b-9f8d-1eca2a47cb15 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:22.837634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485b571-716c-4cc4-92b4-9ae8c4ba18e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:22.838771 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485b571-716c-4cc4-92b4-9ae8c4ba18e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:22.839268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485b571-716c-4cc4-92b4-9ae8c4ba18e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:22.839930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485b571-716c-4cc4-92b4-9ae8c4ba18e3 admin admin] PATH_INFO: `/v3/users/24ea439669ed4b5c92e7e9368aaace8b` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:22.847530 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485b571-716c-4cc4-92b4-9ae8c4ba18e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24ea439669ed4b5c92e7e9368aaace8b)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:22.855122 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485b571-716c-4cc4-92b4-9ae8c4ba18e3 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:23.107009 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-4485b571-716c-4cc4-92b4-9ae8c4ba18e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24ea439669ed4b5c92e7e9368aaace8b'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:23.109426 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:23.130766 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-4485b571-716c-4cc4-92b4-9ae8c4ba18e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24ea439669ed4b5c92e7e9368aaace8b'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:23.133188 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-4485b571-716c-4cc4-92b4-9ae8c4ba18e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24ea439669ed4b5c92e7e9368aaace8b'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:23.241509 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-4485b571-716c-4cc4-92b4-9ae8c4ba18e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24ea439669ed4b5c92e7e9368aaace8b'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:23.268469 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:23.290413 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-4485b571-716c-4cc4-92b4-9ae8c4ba18e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24ea439669ed4b5c92e7e9368aaace8b'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:23.305807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 970/1963] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:07:22 2022] DELETE /identity/v3/users/24ea439669ed4b5c92e7e9368aaace8b => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:23.307577 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c217f02-db70-45ef-80a3-bf1f75a1cd3f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:23.307889 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c217f02-db70-45ef-80a3-bf1f75a1cd3f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:23.308168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c217f02-db70-45ef-80a3-bf1f75a1cd3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:23.377402 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5c217f02-db70-45ef-80a3-bf1f75a1cd3f None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:23.379473 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5c217f02-db70-45ef-80a3-bf1f75a1cd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:23.522115 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 994/1964] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:23.542428 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485b571-716c-4cc4-92b4-9ae8c4ba18e3 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:23.544651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4485b571-716c-4cc4-92b4-9ae8c4ba18e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:23.651978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485b571-716c-4cc4-92b4-9ae8c4ba18e3 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:23.667163 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e461b23-7650-4573-a3ab-126bb05c6ae2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:23.668236 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e461b23-7650-4573-a3ab-126bb05c6ae2 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:23.668873 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e461b23-7650-4573-a3ab-126bb05c6ae2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:23.672406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e461b23-7650-4573-a3ab-126bb05c6ae2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:23.685238 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e461b23-7650-4573-a3ab-126bb05c6ae2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:23.687600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e461b23-7650-4573-a3ab-126bb05c6ae2 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:23.708771 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 971/1965] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:07:24.646827 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:24.916872 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c217f02-db70-45ef-80a3-bf1f75a1cd3f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:24.918405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5c217f02-db70-45ef-80a3-bf1f75a1cd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:24.937537 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:24.941220 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c217f02-db70-45ef-80a3-bf1f75a1cd3f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:24.953201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b8783-b511-4331-8e58-68241e2047a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:24.954399 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b8783-b511-4331-8e58-68241e2047a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:24.955115 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b8783-b511-4331-8e58-68241e2047a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:24.955424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b8783-b511-4331-8e58-68241e2047a3 admin admin] PATH_INFO: `/v3/projects/dfd04ed2a8a845e7910bf7d24df1faa2` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:24.961234 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b8783-b511-4331-8e58-68241e2047a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfd04ed2a8a845e7910bf7d24df1faa2)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:24.971136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b8783-b511-4331-8e58-68241e2047a3 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:25.320626 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-509b8783-b511-4331-8e58-68241e2047a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfd04ed2a8a845e7910bf7d24df1faa2', 'request_id': 'req-509b8783-b511-4331-8e58-68241e2047a3'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:25.377108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-509b8783-b511-4331-8e58-68241e2047a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfd04ed2a8a845e7910bf7d24df1faa2', 'request_id': 'req-509b8783-b511-4331-8e58-68241e2047a3'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:25.507320 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-509b8783-b511-4331-8e58-68241e2047a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfd04ed2a8a845e7910bf7d24df1faa2', 'request_id': 'req-509b8783-b511-4331-8e58-68241e2047a3'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:25.527482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-509b8783-b511-4331-8e58-68241e2047a3 admin admin] The token cache is being invalidate because project dfd04ed2a8a845e7910bf7d24df1faa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:07:25.527730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-509b8783-b511-4331-8e58-68241e2047a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:25.539196 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-509b8783-b511-4331-8e58-68241e2047a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:25.546894 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 995/1966] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:07:24 2022] DELETE /identity/v3/projects/dfd04ed2a8a845e7910bf7d24df1faa2 => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:25.547087 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e461b23-7650-4573-a3ab-126bb05c6ae2 service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:25.576133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e461b23-7650-4573-a3ab-126bb05c6ae2 service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:25.698101 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b8783-b511-4331-8e58-68241e2047a3 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:25.702512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-509b8783-b511-4331-8e58-68241e2047a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:25.763280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55743a35-4be1-4d76-b0a7-f0939377af16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:25.764321 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55743a35-4be1-4d76-b0a7-f0939377af16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:25.764762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55743a35-4be1-4d76-b0a7-f0939377af16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:25.765194 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55743a35-4be1-4d76-b0a7-f0939377af16 admin admin] PATH_INFO: `/v3/users/7d7d85a3b9474c7f9a92b7b45a518986` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:25.767568 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55743a35-4be1-4d76-b0a7-f0939377af16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d7d85a3b9474c7f9a92b7b45a518986)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:25.770265 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55743a35-4be1-4d76-b0a7-f0939377af16 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:25.952968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b8783-b511-4331-8e58-68241e2047a3 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:26.005548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e069e6d-461a-46a4-aa43-fa5cd9a211c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:26.007789 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e069e6d-461a-46a4-aa43-fa5cd9a211c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:26.008531 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e069e6d-461a-46a4-aa43-fa5cd9a211c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:26.010083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e069e6d-461a-46a4-aa43-fa5cd9a211c2 admin admin] PATH_INFO: `/v3/users/7ed9547368734943916f581193fa51ae` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:26.017071 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e069e6d-461a-46a4-aa43-fa5cd9a211c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ed9547368734943916f581193fa51ae)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:26.027548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e069e6d-461a-46a4-aa43-fa5cd9a211c2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:26.062507 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-55743a35-4be1-4d76-b0a7-f0939377af16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d7d85a3b9474c7f9a92b7b45a518986'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:26.092761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-55743a35-4be1-4d76-b0a7-f0939377af16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d7d85a3b9474c7f9a92b7b45a518986'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:26.094171 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-55743a35-4be1-4d76-b0a7-f0939377af16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d7d85a3b9474c7f9a92b7b45a518986'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:26.219956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-55743a35-4be1-4d76-b0a7-f0939377af16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d7d85a3b9474c7f9a92b7b45a518986'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:26.262249 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-55743a35-4be1-4d76-b0a7-f0939377af16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d7d85a3b9474c7f9a92b7b45a518986'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:26.274775 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 972/1967] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:07:25 2022] DELETE /identity/v3/users/7d7d85a3b9474c7f9a92b7b45a518986 => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:26.360708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4e069e6d-461a-46a4-aa43-fa5cd9a211c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ed9547368734943916f581193fa51ae'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:26.372154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4e069e6d-461a-46a4-aa43-fa5cd9a211c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ed9547368734943916f581193fa51ae'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:26.373219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4e069e6d-461a-46a4-aa43-fa5cd9a211c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ed9547368734943916f581193fa51ae'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:26.383559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:26.404609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4e069e6d-461a-46a4-aa43-fa5cd9a211c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ed9547368734943916f581193fa51ae'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:26.406073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4e069e6d-461a-46a4-aa43-fa5cd9a211c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ed9547368734943916f581193fa51ae'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:26.497387 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 996/1968] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:07:25 2022] DELETE /identity/v3/users/7ed9547368734943916f581193fa51ae => generated 0 bytes in 801 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:26.504327 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99955e-0ce6-4001-abe2-f11f4fda9801 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:26.504827 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99955e-0ce6-4001-abe2-f11f4fda9801 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:26.505296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99955e-0ce6-4001-abe2-f11f4fda9801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:26.552013 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-dc99955e-0ce6-4001-abe2-f11f4fda9801 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:26.555257 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-dc99955e-0ce6-4001-abe2-f11f4fda9801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:26.702088 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 973/1969] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:26.730890 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:26.754975 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e069e6d-461a-46a4-aa43-fa5cd9a211c2 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:26.756802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4e069e6d-461a-46a4-aa43-fa5cd9a211c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:26.943893 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e069e6d-461a-46a4-aa43-fa5cd9a211c2 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:26.951383 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff17acf-b7a7-48d2-ac12-0c6a46970645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:26.954228 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff17acf-b7a7-48d2-ac12-0c6a46970645 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:26.955442 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff17acf-b7a7-48d2-ac12-0c6a46970645 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:26.956101 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff17acf-b7a7-48d2-ac12-0c6a46970645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:26.976275 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff17acf-b7a7-48d2-ac12-0c6a46970645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:26.979622 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff17acf-b7a7-48d2-ac12-0c6a46970645 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:27.002448 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 997/1970] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:07:27.074620 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6574b9f-4803-4cff-b0fb-9a3dffacaab5 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:27.076341 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6574b9f-4803-4cff-b0fb-9a3dffacaab5 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:27.077596 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6574b9f-4803-4cff-b0fb-9a3dffacaab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:27.184945 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-a6574b9f-4803-4cff-b0fb-9a3dffacaab5 None None] MFA Rules not processed for user `8b2daf470f45422a8c1bd5cf62efdd9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:27.187913 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-a6574b9f-4803-4cff-b0fb-9a3dffacaab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:27.392044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 974/1971] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:27.396605 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff17acf-b7a7-48d2-ac12-0c6a46970645 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:27.398708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-dff17acf-b7a7-48d2-ac12-0c6a46970645 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:27.417716 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff17acf-b7a7-48d2-ac12-0c6a46970645 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:27.423880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef39b0-219e-4529-9368-d2a6356c2b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:27.425041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef39b0-219e-4529-9368-d2a6356c2b45 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:27.426286 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef39b0-219e-4529-9368-d2a6356c2b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:27.426791 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef39b0-219e-4529-9368-d2a6356c2b45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:27.430254 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef39b0-219e-4529-9368-d2a6356c2b45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:27.430254 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef39b0-219e-4529-9368-d2a6356c2b45 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:27.678884 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 998/1972] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:07:27 2022] POST /identity/v3/projects => generated 385 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:27.682013 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6574b9f-4803-4cff-b0fb-9a3dffacaab5 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:27.704183 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6574b9f-4803-4cff-b0fb-9a3dffacaab5 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:27.788986 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492eebb6-97d3-4329-9d0a-23c5e20fbfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:27.789732 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492eebb6-97d3-4329-9d0a-23c5e20fbfcf admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:27.790688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492eebb6-97d3-4329-9d0a-23c5e20fbfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:27.791043 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492eebb6-97d3-4329-9d0a-23c5e20fbfcf admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:27.791801 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492eebb6-97d3-4329-9d0a-23c5e20fbfcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:27.794083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492eebb6-97d3-4329-9d0a-23c5e20fbfcf admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:27.879667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:27.950298 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 975/1973] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:07:27 2022] POST /identity/v3/users => generated 299 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:07:27.955541 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef39b0-219e-4529-9368-d2a6356c2b45 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:27.961170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:27.988056 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef39b0-219e-4529-9368-d2a6356c2b45 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:28.001506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492eebb6-97d3-4329-9d0a-23c5e20fbfcf admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:28.010622 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2b290-8791-44b4-9766-946f1ca427e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:28.011343 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2b290-8791-44b4-9766-946f1ca427e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:28.011667 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2b290-8791-44b4-9766-946f1ca427e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:28.012018 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2b290-8791-44b4-9766-946f1ca427e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:28.012747 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b2b290-8791-44b4-9766-946f1ca427e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:28.014452 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b2b290-8791-44b4-9766-946f1ca427e8 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:28.040017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-492eebb6-97d3-4329-9d0a-23c5e20fbfcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:28.056246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 999/1974] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:28.062225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2b290-8791-44b4-9766-946f1ca427e8 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:28.065935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e9b2b290-8791-44b4-9766-946f1ca427e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:28.086361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b2b290-8791-44b4-9766-946f1ca427e8 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:28.092915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d3a60-8202-49a6-89f5-81a6002d8fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:28.094268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d3a60-8202-49a6-89f5-81a6002d8fc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:28.094558 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d3a60-8202-49a6-89f5-81a6002d8fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:28.095021 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d3a60-8202-49a6-89f5-81a6002d8fc0 admin admin] PATH_INFO: `/v3/projects/f7c6d9785c014fb880ceb507f0e86fe3` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:28.097954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d3a60-8202-49a6-89f5-81a6002d8fc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7c6d9785c014fb880ceb507f0e86fe3)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:28.102781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d3a60-8202-49a6-89f5-81a6002d8fc0 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:28.155606 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492eebb6-97d3-4329-9d0a-23c5e20fbfcf admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:28.179201 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ba54d-22a7-4340-8ae0-9236b71324bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:28.185103 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ba54d-22a7-4340-8ae0-9236b71324bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:28.186374 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ba54d-22a7-4340-8ae0-9236b71324bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:28.191434 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ba54d-22a7-4340-8ae0-9236b71324bd admin admin] PATH_INFO: `/v3/projects/93a5c2fb5015413a8219480fee1dc4cb` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:28.197564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ba54d-22a7-4340-8ae0-9236b71324bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=93a5c2fb5015413a8219480fee1dc4cb)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:28.199921 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ba54d-22a7-4340-8ae0-9236b71324bd admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:28.361955 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-495d3a60-8202-49a6-89f5-81a6002d8fc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7c6d9785c014fb880ceb507f0e86fe3', 'request_id': 'req-495d3a60-8202-49a6-89f5-81a6002d8fc0'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:28.381033 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-495d3a60-8202-49a6-89f5-81a6002d8fc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7c6d9785c014fb880ceb507f0e86fe3', 'request_id': 'req-495d3a60-8202-49a6-89f5-81a6002d8fc0'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:28.477091 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-097ba54d-22a7-4340-8ae0-9236b71324bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93a5c2fb5015413a8219480fee1dc4cb', 'request_id': 'req-097ba54d-22a7-4340-8ae0-9236b71324bd'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:28.490916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-097ba54d-22a7-4340-8ae0-9236b71324bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93a5c2fb5015413a8219480fee1dc4cb', 'request_id': 'req-097ba54d-22a7-4340-8ae0-9236b71324bd'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:28.511059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-097ba54d-22a7-4340-8ae0-9236b71324bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93a5c2fb5015413a8219480fee1dc4cb', 'request_id': 'req-097ba54d-22a7-4340-8ae0-9236b71324bd'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:28.547620 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-495d3a60-8202-49a6-89f5-81a6002d8fc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7c6d9785c014fb880ceb507f0e86fe3', 'request_id': 'req-495d3a60-8202-49a6-89f5-81a6002d8fc0'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:28.565208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-495d3a60-8202-49a6-89f5-81a6002d8fc0 admin admin] The token cache is being invalidate because project f7c6d9785c014fb880ceb507f0e86fe3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:07:28.565596 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-495d3a60-8202-49a6-89f5-81a6002d8fc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:28.569358 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-495d3a60-8202-49a6-89f5-81a6002d8fc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:28.573694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1000/1975] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:07:28 2022] DELETE /identity/v3/projects/f7c6d9785c014fb880ceb507f0e86fe3 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:28.573694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d3a60-8202-49a6-89f5-81a6002d8fc0 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:28.577647 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-495d3a60-8202-49a6-89f5-81a6002d8fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:28.660071 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d3a60-8202-49a6-89f5-81a6002d8fc0 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:28.663801 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-097ba54d-22a7-4340-8ae0-9236b71324bd admin admin] The token cache is being invalidate because project 93a5c2fb5015413a8219480fee1dc4cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:07:28.664289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-097ba54d-22a7-4340-8ae0-9236b71324bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:28.666249 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-097ba54d-22a7-4340-8ae0-9236b71324bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:28.670108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 976/1976] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:07:27 2022] DELETE /identity/v3/projects/93a5c2fb5015413a8219480fee1dc4cb => generated 0 bytes in 687 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:28.673546 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ba54d-22a7-4340-8ae0-9236b71324bd admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:28.677074 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-097ba54d-22a7-4340-8ae0-9236b71324bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:28.787119 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd582bb-b843-4b5e-9470-1dbf25f0fdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:28.788764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd582bb-b843-4b5e-9470-1dbf25f0fdd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:28.789184 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd582bb-b843-4b5e-9470-1dbf25f0fdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:28.789482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd582bb-b843-4b5e-9470-1dbf25f0fdd7 admin admin] PATH_INFO: `/v3/projects/9e6f2eeeb8d84656824effb901fc9499/users/f2526b051be64aa0845f665d73bf9bf6/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:28.814082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd582bb-b843-4b5e-9470-1dbf25f0fdd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e6f2eeeb8d84656824effb901fc9499, user_id=f2526b051be64aa0845f665d73bf9bf6, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:28.817172 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd582bb-b843-4b5e-9470-1dbf25f0fdd7 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:28.843813 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ba54d-22a7-4340-8ae0-9236b71324bd admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:28.850045 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e9e29-f677-4865-a0b1-349c90cec8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:28.851088 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e9e29-f677-4865-a0b1-349c90cec8bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:28.851509 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e9e29-f677-4865-a0b1-349c90cec8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:28.851992 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e9e29-f677-4865-a0b1-349c90cec8bc admin admin] PATH_INFO: `/v3/users/7c13f09f637c4ccd87e0b5036a1602e4` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:28.854208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e9e29-f677-4865-a0b1-349c90cec8bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c13f09f637c4ccd87e0b5036a1602e4)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:28.857201 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e9e29-f677-4865-a0b1-349c90cec8bc admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:28.930441 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1001/1977] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:28 2022] PUT /identity/v3/projects/9e6f2eeeb8d84656824effb901fc9499/users/f2526b051be64aa0845f665d73bf9bf6/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:28.932290 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c7bb6-3ca4-4750-9e5a-36f2f98b8bb8 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:28.932690 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c7bb6-3ca4-4750-9e5a-36f2f98b8bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:28.933085 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c7bb6-3ca4-4750-9e5a-36f2f98b8bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:28.983119 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-758c7bb6-3ca4-4750-9e5a-36f2f98b8bb8 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:28.985246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-758c7bb6-3ca4-4750-9e5a-36f2f98b8bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:29.072266 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1002/1978] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:29.074229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c7bb6-3ca4-4750-9e5a-36f2f98b8bb8 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:29.086686 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-758c7bb6-3ca4-4750-9e5a-36f2f98b8bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:29.102718 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c7bb6-3ca4-4750-9e5a-36f2f98b8bb8 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:29.110323 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954e623-bae2-45a3-b1a8-c80f7b8e5f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:29.111225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954e623-bae2-45a3-b1a8-c80f7b8e5f0e admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:29.111721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954e623-bae2-45a3-b1a8-c80f7b8e5f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:29.112213 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954e623-bae2-45a3-b1a8-c80f7b8e5f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:29.113183 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954e623-bae2-45a3-b1a8-c80f7b8e5f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:29.114950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954e623-bae2-45a3-b1a8-c80f7b8e5f0e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:29.115989 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-2a3e9e29-f677-4865-a0b1-349c90cec8bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c13f09f637c4ccd87e0b5036a1602e4'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:29.130968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1003/1979] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:29.130968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3470ac3-1d56-4964-910c-be44acb1cfcf None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:29.130968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3470ac3-1d56-4964-910c-be44acb1cfcf None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:29.132197 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3470ac3-1d56-4964-910c-be44acb1cfcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:29.141338 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-2a3e9e29-f677-4865-a0b1-349c90cec8bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c13f09f637c4ccd87e0b5036a1602e4'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:29.142334 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-2a3e9e29-f677-4865-a0b1-349c90cec8bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c13f09f637c4ccd87e0b5036a1602e4'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:29.151896 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:29.166464 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f3470ac3-1d56-4964-910c-be44acb1cfcf None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:29.167962 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f3470ac3-1d56-4964-910c-be44acb1cfcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:29.176858 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1004/1980] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:29.177708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3470ac3-1d56-4964-910c-be44acb1cfcf None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:29.188948 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3470ac3-1d56-4964-910c-be44acb1cfcf None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:29.195000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3e929-1828-4914-af40-51fc4bed654e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:29.195940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3e929-1828-4914-af40-51fc4bed654e admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:29.196224 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3e929-1828-4914-af40-51fc4bed654e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:29.196529 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3e929-1828-4914-af40-51fc4bed654e admin admin] PATH_INFO: `/v3/projects/9e6f2eeeb8d84656824effb901fc9499/users/f2526b051be64aa0845f665d73bf9bf6/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:29.199942 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3e929-1828-4914-af40-51fc4bed654e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e6f2eeeb8d84656824effb901fc9499, user_id=f2526b051be64aa0845f665d73bf9bf6, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:29.203980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3e929-1828-4914-af40-51fc4bed654e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:29.222044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1005/1981] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:29 2022] PUT /identity/v3/projects/9e6f2eeeb8d84656824effb901fc9499/users/f2526b051be64aa0845f665d73bf9bf6/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:29.223236 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3e929-1828-4914-af40-51fc4bed654e admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:29.278417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-2a3e9e29-f677-4865-a0b1-349c90cec8bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c13f09f637c4ccd87e0b5036a1602e4'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:29.304196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-2a3e9e29-f677-4865-a0b1-349c90cec8bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c13f09f637c4ccd87e0b5036a1602e4'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:29.314406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 977/1982] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:07:28 2022] DELETE /identity/v3/users/7c13f09f637c4ccd87e0b5036a1602e4 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:29.387432 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3e929-1828-4914-af40-51fc4bed654e admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:29.490142 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0fa72-2ed1-43ce-ae2e-77284d5e8bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:29.490925 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0fa72-2ed1-43ce-ae2e-77284d5e8bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:29.491296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0fa72-2ed1-43ce-ae2e-77284d5e8bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:29.491605 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0fa72-2ed1-43ce-ae2e-77284d5e8bd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:29.493076 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0fa72-2ed1-43ce-ae2e-77284d5e8bd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:07:29.493273 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0fa72-2ed1-43ce-ae2e-77284d5e8bd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:29.495993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0fa72-2ed1-43ce-ae2e-77284d5e8bd0 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:29.516520 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1006/1983] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:07:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:07:29.523795 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e9e29-f677-4865-a0b1-349c90cec8bc admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:29.523795 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-2a3e9e29-f677-4865-a0b1-349c90cec8bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:29.561747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e9e29-f677-4865-a0b1-349c90cec8bc admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:29.578072 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a868a-ddc4-4634-afd5-7257f0e57d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:29.578783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a868a-ddc4-4634-afd5-7257f0e57d2e admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:29.579203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a868a-ddc4-4634-afd5-7257f0e57d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:29.579482 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a868a-ddc4-4634-afd5-7257f0e57d2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:29.580426 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4a868a-ddc4-4634-afd5-7257f0e57d2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:29.582745 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4a868a-ddc4-4634-afd5-7257f0e57d2e admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:29.709866 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 978/1984] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:07:29 2022] POST /identity/v3/projects => generated 375 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:29.716878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0fa72-2ed1-43ce-ae2e-77284d5e8bd0 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:29.730504 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0fa72-2ed1-43ce-ae2e-77284d5e8bd0 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:29.800146 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:29.868810 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a0b93-58aa-42a3-bcbc-7b40d210c3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:29.869458 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a0b93-58aa-42a3-bcbc-7b40d210c3ea admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:29.869922 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a0b93-58aa-42a3-bcbc-7b40d210c3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:29.870210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70a0b93-58aa-42a3-bcbc-7b40d210c3ea admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:29.870993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a0b93-58aa-42a3-bcbc-7b40d210c3ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:29.888469 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70a0b93-58aa-42a3-bcbc-7b40d210c3ea admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:30.026124 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a868a-ddc4-4634-afd5-7257f0e57d2e admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:30.027094 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1007/1985] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:07:29 2022] POST /identity/v3/users => generated 288 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:30.045916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a868a-ddc4-4634-afd5-7257f0e57d2e admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:30.059420 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2fcacb-5d32-420b-b5cf-e4ea2474b622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:30.060253 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fcacb-5d32-420b-b5cf-e4ea2474b622 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:30.061788 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fcacb-5d32-420b-b5cf-e4ea2474b622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:30.063009 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fcacb-5d32-420b-b5cf-e4ea2474b622 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:30.066872 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2fcacb-5d32-420b-b5cf-e4ea2474b622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:30.068927 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2fcacb-5d32-420b-b5cf-e4ea2474b622 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:30.110710 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a0b93-58aa-42a3-bcbc-7b40d210c3ea admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:30.129956 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70a0b93-58aa-42a3-bcbc-7b40d210c3ea admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:30.130791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 979/1986] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:30.146027 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c6cf6-01c9-4b8d-a39d-305f440d3647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:30.146691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c6cf6-01c9-4b8d-a39d-305f440d3647 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:30.146976 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c6cf6-01c9-4b8d-a39d-305f440d3647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:30.147261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c6cf6-01c9-4b8d-a39d-305f440d3647 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:30.148664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c6cf6-01c9-4b8d-a39d-305f440d3647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:30.152009 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c6cf6-01c9-4b8d-a39d-305f440d3647 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:30.190399 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1008/1987] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:30.192422 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2fcacb-5d32-420b-b5cf-e4ea2474b622 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:30.216115 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2fcacb-5d32-420b-b5cf-e4ea2474b622 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:30.238986 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07848653-7d38-4fbc-8474-f41aebc93cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:30.239735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07848653-7d38-4fbc-8474-f41aebc93cb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:30.240074 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07848653-7d38-4fbc-8474-f41aebc93cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:30.240357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07848653-7d38-4fbc-8474-f41aebc93cb2 admin admin] PATH_INFO: `/v3/projects/3960392fe03e4cc995dd6fc7841382bc/users/31c682d0477d4be3b092a7fced6ce4d1/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:30.273991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07848653-7d38-4fbc-8474-f41aebc93cb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3960392fe03e4cc995dd6fc7841382bc, user_id=31c682d0477d4be3b092a7fced6ce4d1, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:30.277557 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07848653-7d38-4fbc-8474-f41aebc93cb2 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:30.411491 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 980/1988] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:30 2022] PUT /identity/v3/projects/3960392fe03e4cc995dd6fc7841382bc/users/31c682d0477d4be3b092a7fced6ce4d1/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:30.429436 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d29aa4-3e71-4b3f-aae7-2e0398fd3e4c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:30.429800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d29aa4-3e71-4b3f-aae7-2e0398fd3e4c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:30.430170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d29aa4-3e71-4b3f-aae7-2e0398fd3e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:30.519742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-c0d29aa4-3e71-4b3f-aae7-2e0398fd3e4c None None] MFA Rules not processed for user `31c682d0477d4be3b092a7fced6ce4d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:30.519742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c0d29aa4-3e71-4b3f-aae7-2e0398fd3e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:30.638199 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1009/1989] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:30.662799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07848653-7d38-4fbc-8474-f41aebc93cb2 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:30.666492 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-07848653-7d38-4fbc-8474-f41aebc93cb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:30.742325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d29aa4-3e71-4b3f-aae7-2e0398fd3e4c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:30.744222 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c0d29aa4-3e71-4b3f-aae7-2e0398fd3e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:30.927998 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07848653-7d38-4fbc-8474-f41aebc93cb2 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:30.942643 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f391fabf-66f1-40b5-9698-c394c0cc12db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:30.944022 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391fabf-66f1-40b5-9698-c394c0cc12db service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:30.944627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391fabf-66f1-40b5-9698-c394c0cc12db service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:30.945209 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f391fabf-66f1-40b5-9698-c394c0cc12db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:30.985753 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d29aa4-3e71-4b3f-aae7-2e0398fd3e4c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:30.988962 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f391fabf-66f1-40b5-9698-c394c0cc12db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:30.991654 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f31083-47d1-44f9-a298-7e0a8c2fb53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:30.992487 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f31083-47d1-44f9-a298-7e0a8c2fb53b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:30.992866 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f31083-47d1-44f9-a298-7e0a8c2fb53b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:30.993240 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f31083-47d1-44f9-a298-7e0a8c2fb53b admin admin] PATH_INFO: `/v3/projects/54a9ade34599442fb03bcd3215fc0121` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:30.994801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f31083-47d1-44f9-a298-7e0a8c2fb53b admin admin] RBAC: Authorizing `identity:delete_project(project_id=54a9ade34599442fb03bcd3215fc0121)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:30.997538 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f31083-47d1-44f9-a298-7e0a8c2fb53b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:31.003624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f391fabf-66f1-40b5-9698-c394c0cc12db service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:31.032380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 981/1990] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:07:31.184764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:31.234400 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d8f31083-47d1-44f9-a298-7e0a8c2fb53b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54a9ade34599442fb03bcd3215fc0121', 'request_id': 'req-d8f31083-47d1-44f9-a298-7e0a8c2fb53b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:31.252660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d8f31083-47d1-44f9-a298-7e0a8c2fb53b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54a9ade34599442fb03bcd3215fc0121', 'request_id': 'req-d8f31083-47d1-44f9-a298-7e0a8c2fb53b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:31.380254 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d8f31083-47d1-44f9-a298-7e0a8c2fb53b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54a9ade34599442fb03bcd3215fc0121', 'request_id': 'req-d8f31083-47d1-44f9-a298-7e0a8c2fb53b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:31.408579 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d8f31083-47d1-44f9-a298-7e0a8c2fb53b admin admin] The token cache is being invalidate because project 54a9ade34599442fb03bcd3215fc0121 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:07:31.415146 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d8f31083-47d1-44f9-a298-7e0a8c2fb53b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:31.419275 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d8f31083-47d1-44f9-a298-7e0a8c2fb53b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:31.421557 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1010/1991] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:07:30 2022] DELETE /identity/v3/projects/54a9ade34599442fb03bcd3215fc0121 => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:07:31.427079 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1adbeb-c780-4fba-a498-0fe8d627fe9d None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:31.427678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1adbeb-c780-4fba-a498-0fe8d627fe9d None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:31.427943 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1adbeb-c780-4fba-a498-0fe8d627fe9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:31.483175 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-1e1adbeb-c780-4fba-a498-0fe8d627fe9d None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:31.485497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-1e1adbeb-c780-4fba-a498-0fe8d627fe9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:31.629333 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 982/1992] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:07:31.639018 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d24972-5aab-4a14-96e6-5faea0a97561 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:31.639308 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d24972-5aab-4a14-96e6-5faea0a97561 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:31.639559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d24972-5aab-4a14-96e6-5faea0a97561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:31.678846 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-a2d24972-5aab-4a14-96e6-5faea0a97561 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:31.680394 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a2d24972-5aab-4a14-96e6-5faea0a97561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:31.694054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1011/1993] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:31.696411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1adbeb-c780-4fba-a498-0fe8d627fe9d None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:31.743061 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1adbeb-c780-4fba-a498-0fe8d627fe9d None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:31.754290 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be640b-e654-4f29-8332-e44c6fb3bb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:31.755495 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be640b-e654-4f29-8332-e44c6fb3bb82 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:31.756024 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be640b-e654-4f29-8332-e44c6fb3bb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:31.756460 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be640b-e654-4f29-8332-e44c6fb3bb82 admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:31.757547 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be640b-e654-4f29-8332-e44c6fb3bb82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:07:31.758198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be640b-e654-4f29-8332-e44c6fb3bb82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:31.762341 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be640b-e654-4f29-8332-e44c6fb3bb82 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:31.781523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 983/1994] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:07:31.785403 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d24972-5aab-4a14-96e6-5faea0a97561 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:31.799750 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d24972-5aab-4a14-96e6-5faea0a97561 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:31.807932 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96256881-3979-4b1a-902b-8641684f841d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:31.809117 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96256881-3979-4b1a-902b-8641684f841d admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:31.809587 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96256881-3979-4b1a-902b-8641684f841d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:31.810056 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96256881-3979-4b1a-902b-8641684f841d admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:31.810974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96256881-3979-4b1a-902b-8641684f841d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:31.813669 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96256881-3979-4b1a-902b-8641684f841d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:31.943219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1012/1995] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:07:31 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:07:31.944545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be640b-e654-4f29-8332-e44c6fb3bb82 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:31.967654 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be640b-e654-4f29-8332-e44c6fb3bb82 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:32.090017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a9471-03b0-4fec-9cac-6b7776cccb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:32.090983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a9471-03b0-4fec-9cac-6b7776cccb02 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:32.092195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a9471-03b0-4fec-9cac-6b7776cccb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:32.092479 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a9471-03b0-4fec-9cac-6b7776cccb02 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:32.093207 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a9471-03b0-4fec-9cac-6b7776cccb02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:32.102601 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a9471-03b0-4fec-9cac-6b7776cccb02 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:32.308408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96256881-3979-4b1a-902b-8641684f841d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:32.314018 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 984/1996] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:07:31 2022] POST /identity/v3/users => generated 290 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:32.324929 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96256881-3979-4b1a-902b-8641684f841d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:32.356813 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5536ce7-504f-44f9-a862-3662c00bb825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:32.357582 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5536ce7-504f-44f9-a862-3662c00bb825 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:32.357858 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5536ce7-504f-44f9-a862-3662c00bb825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:32.358173 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5536ce7-504f-44f9-a862-3662c00bb825 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:32.358902 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5536ce7-504f-44f9-a862-3662c00bb825 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:32.360594 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5536ce7-504f-44f9-a862-3662c00bb825 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:32.396863 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1013/1997] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:07:32.405432 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a9471-03b0-4fec-9cac-6b7776cccb02 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:32.424498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a9471-03b0-4fec-9cac-6b7776cccb02 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:32.434378 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f3a58-69f5-49d9-adb9-0fca76b3b91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:32.438688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f3a58-69f5-49d9-adb9-0fca76b3b91f admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:32.439049 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f3a58-69f5-49d9-adb9-0fca76b3b91f admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:32.440384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f3a58-69f5-49d9-adb9-0fca76b3b91f admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:32.441369 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f3a58-69f5-49d9-adb9-0fca76b3b91f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:32.450570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f3a58-69f5-49d9-adb9-0fca76b3b91f admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:32.484650 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 985/1998] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:32.489747 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5536ce7-504f-44f9-a862-3662c00bb825 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:32.501870 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5536ce7-504f-44f9-a862-3662c00bb825 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:32.511837 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e51c78-f722-4ef9-b2ff-3a18199908d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:32.512549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e51c78-f722-4ef9-b2ff-3a18199908d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:32.512839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e51c78-f722-4ef9-b2ff-3a18199908d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:32.513125 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e51c78-f722-4ef9-b2ff-3a18199908d3 admin admin] PATH_INFO: `/v3/projects/2a36fab2cc8c419ca482e8131562f7bf/users/b33911db94b84703a65d786c3e05d9d9/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:32.539011 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e51c78-f722-4ef9-b2ff-3a18199908d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a36fab2cc8c419ca482e8131562f7bf, user_id=b33911db94b84703a65d786c3e05d9d9, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:32.543721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e51c78-f722-4ef9-b2ff-3a18199908d3 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:32.724439 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:32.738023 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1014/1999] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:32 2022] PUT /identity/v3/projects/2a36fab2cc8c419ca482e8131562f7bf/users/b33911db94b84703a65d786c3e05d9d9/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:32.748191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920cc38e-5a87-486a-9f28-b836b55c5160 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:32.748886 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920cc38e-5a87-486a-9f28-b836b55c5160 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:32.749476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920cc38e-5a87-486a-9f28-b836b55c5160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:32.799731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-920cc38e-5a87-486a-9f28-b836b55c5160 None None] MFA Rules not processed for user `b33911db94b84703a65d786c3e05d9d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:32.802087 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-920cc38e-5a87-486a-9f28-b836b55c5160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:32.912024 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 986/2000] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:32.913094 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e51c78-f722-4ef9-b2ff-3a18199908d3 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:33.006832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e51c78-f722-4ef9-b2ff-3a18199908d3 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:33.013961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96902c72-d634-4f3b-ad08-46c633e0a820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:33.015589 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96902c72-d634-4f3b-ad08-46c633e0a820 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:33.015834 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96902c72-d634-4f3b-ad08-46c633e0a820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:33.015926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96902c72-d634-4f3b-ad08-46c633e0a820 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:33.017712 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96902c72-d634-4f3b-ad08-46c633e0a820 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:33.024305 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96902c72-d634-4f3b-ad08-46c633e0a820 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:33.148517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1015/2001] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:07:32 2022] POST /identity/v3/projects => generated 381 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:33.149624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920cc38e-5a87-486a-9f28-b836b55c5160 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:33.161193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920cc38e-5a87-486a-9f28-b836b55c5160 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:33.228665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77fe1bd-6f5f-4ffa-8224-a7f4eff39b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:33.229306 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fe1bd-6f5f-4ffa-8224-a7f4eff39b0d admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:33.229588 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fe1bd-6f5f-4ffa-8224-a7f4eff39b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:33.229865 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77fe1bd-6f5f-4ffa-8224-a7f4eff39b0d admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:33.230583 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77fe1bd-6f5f-4ffa-8224-a7f4eff39b0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:33.232510 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77fe1bd-6f5f-4ffa-8224-a7f4eff39b0d admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:33.376864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 987/2002] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:07:33 2022] POST /identity/v3/users => generated 297 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:07:33.381276 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96902c72-d634-4f3b-ad08-46c633e0a820 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:33.398445 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96902c72-d634-4f3b-ad08-46c633e0a820 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:33.415074 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705a1da-e07f-443f-9176-af06799d1019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:33.416331 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705a1da-e07f-443f-9176-af06799d1019 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:33.417013 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705a1da-e07f-443f-9176-af06799d1019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:33.417126 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b705a1da-e07f-443f-9176-af06799d1019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:33.418259 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b705a1da-e07f-443f-9176-af06799d1019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:33.420102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b705a1da-e07f-443f-9176-af06799d1019 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:33.440858 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1016/2003] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:33.445734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77fe1bd-6f5f-4ffa-8224-a7f4eff39b0d admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:33.458697 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77fe1bd-6f5f-4ffa-8224-a7f4eff39b0d admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:33.463556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abb905-8d19-467a-abe5-854425b9a3e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:33.464259 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abb905-8d19-467a-abe5-854425b9a3e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:33.464546 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abb905-8d19-467a-abe5-854425b9a3e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:33.464829 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05abb905-8d19-467a-abe5-854425b9a3e1 admin admin] PATH_INFO: `/v3/projects/28b0840f5f994ff9841b62ebbc30246a/users/1dd659c2d1cb4926a23b112a24aa20c2/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:33.491290 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05abb905-8d19-467a-abe5-854425b9a3e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28b0840f5f994ff9841b62ebbc30246a, user_id=1dd659c2d1cb4926a23b112a24aa20c2, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:33.494662 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05abb905-8d19-467a-abe5-854425b9a3e1 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:33.659500 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 988/2004] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:33 2022] PUT /identity/v3/projects/28b0840f5f994ff9841b62ebbc30246a/users/1dd659c2d1cb4926a23b112a24aa20c2/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:07:33.664662 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705a1da-e07f-443f-9176-af06799d1019 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:33.682056 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b705a1da-e07f-443f-9176-af06799d1019 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:33.758180 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d9249f-414e-4983-9492-ce0998cc0304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:33.758826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9249f-414e-4983-9492-ce0998cc0304 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:33.759147 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9249f-414e-4983-9492-ce0998cc0304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:33.759428 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d9249f-414e-4983-9492-ce0998cc0304 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:33.760290 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9249f-414e-4983-9492-ce0998cc0304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:33.762132 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d9249f-414e-4983-9492-ce0998cc0304 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:33.773117 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1017/2005] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:33.777333 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abb905-8d19-467a-abe5-854425b9a3e1 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:33.790740 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05abb905-8d19-467a-abe5-854425b9a3e1 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:33.810895 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8257e3e-04c5-4ea4-8d0a-bb726a2cb9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:33.811861 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8257e3e-04c5-4ea4-8d0a-bb726a2cb9f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:33.812562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8257e3e-04c5-4ea4-8d0a-bb726a2cb9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:33.812944 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8257e3e-04c5-4ea4-8d0a-bb726a2cb9f6 admin admin] PATH_INFO: `/v3/projects/28b0840f5f994ff9841b62ebbc30246a/users/1dd659c2d1cb4926a23b112a24aa20c2/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:33.815675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8257e3e-04c5-4ea4-8d0a-bb726a2cb9f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28b0840f5f994ff9841b62ebbc30246a, user_id=1dd659c2d1cb4926a23b112a24aa20c2, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:33.818635 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8257e3e-04c5-4ea4-8d0a-bb726a2cb9f6 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:33.832873 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 989/2006] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:33 2022] PUT /identity/v3/projects/28b0840f5f994ff9841b62ebbc30246a/users/1dd659c2d1cb4926a23b112a24aa20c2/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:33.846088 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136341ba-19f5-4564-a8b1-a86fbf7e5823 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:33.846430 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136341ba-19f5-4564-a8b1-a86fbf7e5823 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:33.855443 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136341ba-19f5-4564-a8b1-a86fbf7e5823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:33.885115 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-136341ba-19f5-4564-a8b1-a86fbf7e5823 None None] MFA Rules not processed for user `1dd659c2d1cb4926a23b112a24aa20c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:33.887490 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-136341ba-19f5-4564-a8b1-a86fbf7e5823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:33.986464 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1018/2007] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:33 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:33.998806 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8257e3e-04c5-4ea4-8d0a-bb726a2cb9f6 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:34.003355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-a8257e3e-04c5-4ea4-8d0a-bb726a2cb9f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:34.101961 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8257e3e-04c5-4ea4-8d0a-bb726a2cb9f6 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:34.107051 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4f5a4e-835c-4fa8-9867-05bb56b5c2b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:34.107857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f5a4e-835c-4fa8-9867-05bb56b5c2b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:34.108246 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f5a4e-835c-4fa8-9867-05bb56b5c2b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:34.108631 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4f5a4e-835c-4fa8-9867-05bb56b5c2b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:34.120013 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4f5a4e-835c-4fa8-9867-05bb56b5c2b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:34.122082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4f5a4e-835c-4fa8-9867-05bb56b5c2b9 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:34.123014 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:34.134822 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 990/2008] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:33 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:07:34.517069 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:35.730881 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:35.996055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136341ba-19f5-4564-a8b1-a86fbf7e5823 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:35.996576 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:36.075107 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136341ba-19f5-4564-a8b1-a86fbf7e5823 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:36.080578 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb43d90-c8d5-4672-9057-f41f8a96ec80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:36.081329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb43d90-c8d5-4672-9057-f41f8a96ec80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:36.081803 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb43d90-c8d5-4672-9057-f41f8a96ec80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:36.082565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb43d90-c8d5-4672-9057-f41f8a96ec80 admin admin] PATH_INFO: `/v3/users/b33911db94b84703a65d786c3e05d9d9` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:36.083409 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb43d90-c8d5-4672-9057-f41f8a96ec80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b33911db94b84703a65d786c3e05d9d9)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:36.086629 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb43d90-c8d5-4672-9057-f41f8a96ec80 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:36.251242 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4eb43d90-c8d5-4672-9057-f41f8a96ec80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b33911db94b84703a65d786c3e05d9d9'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:36.263683 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4eb43d90-c8d5-4672-9057-f41f8a96ec80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b33911db94b84703a65d786c3e05d9d9'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:36.264581 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4eb43d90-c8d5-4672-9057-f41f8a96ec80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b33911db94b84703a65d786c3e05d9d9'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:36.278419 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4eb43d90-c8d5-4672-9057-f41f8a96ec80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b33911db94b84703a65d786c3e05d9d9'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:36.279175 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4eb43d90-c8d5-4672-9057-f41f8a96ec80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b33911db94b84703a65d786c3e05d9d9'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:36.377021 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1019/2009] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:07:35 2022] DELETE /identity/v3/users/b33911db94b84703a65d786c3e05d9d9 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:36.382810 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932aa77b-9fe4-4c3a-865d-d089313ee6dd None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:36.383168 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932aa77b-9fe4-4c3a-865d-d089313ee6dd None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:36.383503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932aa77b-9fe4-4c3a-865d-d089313ee6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:36.407202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-932aa77b-9fe4-4c3a-865d-d089313ee6dd None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:36.408999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-932aa77b-9fe4-4c3a-865d-d089313ee6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:36.486716 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 991/2010] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:36.504808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb43d90-c8d5-4672-9057-f41f8a96ec80 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:36.506306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4eb43d90-c8d5-4672-9057-f41f8a96ec80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:36.578459 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb43d90-c8d5-4672-9057-f41f8a96ec80 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:36.582508 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21654c07-d073-4ae4-9df1-c7e7562c1aec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:36.583287 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21654c07-d073-4ae4-9df1-c7e7562c1aec service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:36.583408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21654c07-d073-4ae4-9df1-c7e7562c1aec service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:36.583670 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21654c07-d073-4ae4-9df1-c7e7562c1aec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:36.610623 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21654c07-d073-4ae4-9df1-c7e7562c1aec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:36.610623 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21654c07-d073-4ae4-9df1-c7e7562c1aec service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:36.610623 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1020/2011] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:07:37.303331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932aa77b-9fe4-4c3a-865d-d089313ee6dd None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:37.306030 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-932aa77b-9fe4-4c3a-865d-d089313ee6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:37.325138 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932aa77b-9fe4-4c3a-865d-d089313ee6dd None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:37.331501 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb70c0-aaf5-4140-88a6-c00547c1465b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:37.332857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb70c0-aaf5-4140-88a6-c00547c1465b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:37.333195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb70c0-aaf5-4140-88a6-c00547c1465b admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:37.333823 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb70c0-aaf5-4140-88a6-c00547c1465b admin admin] PATH_INFO: `/v3/projects/2a36fab2cc8c419ca482e8131562f7bf` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:37.337973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb70c0-aaf5-4140-88a6-c00547c1465b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a36fab2cc8c419ca482e8131562f7bf)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:37.339406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb70c0-aaf5-4140-88a6-c00547c1465b admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:37.485969 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-7ceb70c0-aaf5-4140-88a6-c00547c1465b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a36fab2cc8c419ca482e8131562f7bf', 'request_id': 'req-7ceb70c0-aaf5-4140-88a6-c00547c1465b'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:37.495116 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-7ceb70c0-aaf5-4140-88a6-c00547c1465b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a36fab2cc8c419ca482e8131562f7bf', 'request_id': 'req-7ceb70c0-aaf5-4140-88a6-c00547c1465b'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:37.502710 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-7ceb70c0-aaf5-4140-88a6-c00547c1465b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a36fab2cc8c419ca482e8131562f7bf', 'request_id': 'req-7ceb70c0-aaf5-4140-88a6-c00547c1465b'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:37.582852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-7ceb70c0-aaf5-4140-88a6-c00547c1465b admin admin] The token cache is being invalidate because project 2a36fab2cc8c419ca482e8131562f7bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:07:37.582852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-7ceb70c0-aaf5-4140-88a6-c00547c1465b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:37.583486 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-7ceb70c0-aaf5-4140-88a6-c00547c1465b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:37.585839 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 992/2012] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:07:37 2022] DELETE /identity/v3/projects/2a36fab2cc8c419ca482e8131562f7bf => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:37.590330 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21654c07-d073-4ae4-9df1-c7e7562c1aec service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:37.607748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21654c07-d073-4ae4-9df1-c7e7562c1aec service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:37.613532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:37.710945 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:37.808738 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1a19f-a3d7-4de8-8f03-b7cb8711db73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:37.809439 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1a19f-a3d7-4de8-8f03-b7cb8711db73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:37.809727 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1a19f-a3d7-4de8-8f03-b7cb8711db73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:37.810045 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1a19f-a3d7-4de8-8f03-b7cb8711db73 admin admin] PATH_INFO: `/v3/users/1dd659c2d1cb4926a23b112a24aa20c2` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:37.812599 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1a19f-a3d7-4de8-8f03-b7cb8711db73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dd659c2d1cb4926a23b112a24aa20c2)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:37.819222 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1a19f-a3d7-4de8-8f03-b7cb8711db73 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:38.136865 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd0b5c-76b0-4a85-9e27-f5859e864d77 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:38.137457 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd0b5c-76b0-4a85-9e27-f5859e864d77 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:38.140463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd0b5c-76b0-4a85-9e27-f5859e864d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:38.140612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e4a1a19f-a3d7-4de8-8f03-b7cb8711db73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dd659c2d1cb4926a23b112a24aa20c2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:38.147822 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e4a1a19f-a3d7-4de8-8f03-b7cb8711db73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dd659c2d1cb4926a23b112a24aa20c2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:38.148632 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e4a1a19f-a3d7-4de8-8f03-b7cb8711db73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dd659c2d1cb4926a23b112a24aa20c2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:38.184777 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-4fdd0b5c-76b0-4a85-9e27-f5859e864d77 None None] MFA Rules not processed for user `f2526b051be64aa0845f665d73bf9bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:38.187371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4fdd0b5c-76b0-4a85-9e27-f5859e864d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:38.206654 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e4a1a19f-a3d7-4de8-8f03-b7cb8711db73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dd659c2d1cb4926a23b112a24aa20c2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:38.207410 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e4a1a19f-a3d7-4de8-8f03-b7cb8711db73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dd659c2d1cb4926a23b112a24aa20c2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:38.296169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 993/2013] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:07:38.312136 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd0b5c-76b0-4a85-9e27-f5859e864d77 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:38.313616 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4fdd0b5c-76b0-4a85-9e27-f5859e864d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:38.328590 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1021/2014] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:07:37 2022] DELETE /identity/v3/users/1dd659c2d1cb4926a23b112a24aa20c2 => generated 0 bytes in 739 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:38.422137 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd0b5c-76b0-4a85-9e27-f5859e864d77 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:38.479654 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd49acc-c638-49cb-bf6c-ebedec10674d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4abd1512ad224da896c6f2257f9602e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:38.480452 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd49acc-c638-49cb-bf6c-ebedec10674d service glance] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:38.480813 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd49acc-c638-49cb-bf6c-ebedec10674d service glance] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:38.481191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd49acc-c638-49cb-bf6c-ebedec10674d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:38.483034 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-ebd49acc-c638-49cb-bf6c-ebedec10674d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:38.549136 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd49acc-c638-49cb-bf6c-ebedec10674d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:38.551625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd49acc-c638-49cb-bf6c-ebedec10674d service glance] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:38.563363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 994/2015] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:07:38.849372 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1a19f-a3d7-4de8-8f03-b7cb8711db73 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:38.851004 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e4a1a19f-a3d7-4de8-8f03-b7cb8711db73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:38.962371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1a19f-a3d7-4de8-8f03-b7cb8711db73 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:38.973076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60870da2-1636-4ed6-abf8-9b5982c9b589 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:38.973254 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d3d40a-022c-4108-b547-9c04d7520fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:38.973380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60870da2-1636-4ed6-abf8-9b5982c9b589 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:38.973851 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60870da2-1636-4ed6-abf8-9b5982c9b589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:38.974162 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3d40a-022c-4108-b547-9c04d7520fd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:38.974442 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3d40a-022c-4108-b547-9c04d7520fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:38.974736 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d3d40a-022c-4108-b547-9c04d7520fd1 admin admin] PATH_INFO: `/v3/projects/28b0840f5f994ff9841b62ebbc30246a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:38.975925 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3d40a-022c-4108-b547-9c04d7520fd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28b0840f5f994ff9841b62ebbc30246a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:38.978576 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d3d40a-022c-4108-b547-9c04d7520fd1 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:39.002577 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-60870da2-1636-4ed6-abf8-9b5982c9b589 None None] MFA Rules not processed for user `ed8114b7a28741268fdff71c866dbc77`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:39.003832 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-60870da2-1636-4ed6-abf8-9b5982c9b589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:39.086667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 995/2016] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 2 09:07:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:07:39.114182 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:39.138242 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e8d3d40a-022c-4108-b547-9c04d7520fd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28b0840f5f994ff9841b62ebbc30246a', 'request_id': 'req-e8d3d40a-022c-4108-b547-9c04d7520fd1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:39.138384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60870da2-1636-4ed6-abf8-9b5982c9b589 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:39.140119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-60870da2-1636-4ed6-abf8-9b5982c9b589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:39.147564 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e8d3d40a-022c-4108-b547-9c04d7520fd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28b0840f5f994ff9841b62ebbc30246a', 'request_id': 'req-e8d3d40a-022c-4108-b547-9c04d7520fd1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:39.197007 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:39.223546 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60870da2-1636-4ed6-abf8-9b5982c9b589 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:39.261261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e8d3d40a-022c-4108-b547-9c04d7520fd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28b0840f5f994ff9841b62ebbc30246a', 'request_id': 'req-e8d3d40a-022c-4108-b547-9c04d7520fd1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:39.282665 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e8d3d40a-022c-4108-b547-9c04d7520fd1 admin admin] The token cache is being invalidate because project 28b0840f5f994ff9841b62ebbc30246a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:07:39.283203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e8d3d40a-022c-4108-b547-9c04d7520fd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:39.284018 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e8d3d40a-022c-4108-b547-9c04d7520fd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:39.286003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1022/2017] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:07:38 2022] DELETE /identity/v3/projects/28b0840f5f994ff9841b62ebbc30246a => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:39.311128 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215fd6cf-0d84-4719-957a-d6e034347250 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:39.311128 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215fd6cf-0d84-4719-957a-d6e034347250 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:39.311128 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215fd6cf-0d84-4719-957a-d6e034347250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:39.311651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd27792-3b8e-44af-a1da-341fba42516c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff92ef3b9da74ccab3fbc43f852e3ad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:39.323813 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd27792-3b8e-44af-a1da-341fba42516c service swift] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:39.324396 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd27792-3b8e-44af-a1da-341fba42516c service swift] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:39.325531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd27792-3b8e-44af-a1da-341fba42516c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:39.328083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-215fd6cf-0d84-4719-957a-d6e034347250 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:39.330207 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-ecd27792-3b8e-44af-a1da-341fba42516c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:39.330460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-215fd6cf-0d84-4719-957a-d6e034347250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:39.422580 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1023/2018] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:07:39.433747 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb27210-e2f5-4a50-a056-160d545099ae None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:39.434516 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb27210-e2f5-4a50-a056-160d545099ae None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:39.436439 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb27210-e2f5-4a50-a056-160d545099ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:39.456033 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd27792-3b8e-44af-a1da-341fba42516c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:39.457669 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd27792-3b8e-44af-a1da-341fba42516c service swift] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:39.476469 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-ffb27210-e2f5-4a50-a056-160d545099ae None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:39.477824 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ffb27210-e2f5-4a50-a056-160d545099ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:39.490152 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 996/2019] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 2 09:07:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:07:39.501320 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1024/2020] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:39.505158 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd27792-3b8e-44af-a1da-341fba42516c service swift] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:39.538331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd27792-3b8e-44af-a1da-341fba42516c service swift] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:39.551185 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1461f41-00d9-44f5-b550-3eeeecbb92be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:39.552491 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1461f41-00d9-44f5-b550-3eeeecbb92be admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:39.552877 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1461f41-00d9-44f5-b550-3eeeecbb92be admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:39.553111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1461f41-00d9-44f5-b550-3eeeecbb92be admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:39.553828 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1461f41-00d9-44f5-b550-3eeeecbb92be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:07:39.554182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1461f41-00d9-44f5-b550-3eeeecbb92be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:39.557081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1461f41-00d9-44f5-b550-3eeeecbb92be admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:39.580299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 997/2021] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:07:39.584247 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb27210-e2f5-4a50-a056-160d545099ae None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:39.599773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb27210-e2f5-4a50-a056-160d545099ae None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:39.607056 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d9976-c274-4394-bcd6-14a72cc3016a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:39.608354 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d9976-c274-4394-bcd6-14a72cc3016a admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:39.608859 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d9976-c274-4394-bcd6-14a72cc3016a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:39.609532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d9976-c274-4394-bcd6-14a72cc3016a admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:39.611698 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d9976-c274-4394-bcd6-14a72cc3016a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:39.614890 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d9976-c274-4394-bcd6-14a72cc3016a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:39.762063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1025/2022] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:07:39 2022] POST /identity/v3/projects => generated 373 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:07:39.763712 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1461f41-00d9-44f5-b550-3eeeecbb92be admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:39.783225 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1461f41-00d9-44f5-b550-3eeeecbb92be admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:39.861468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:39.904470 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51e74d-20e0-460e-b124-397e04da7c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:39.905520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51e74d-20e0-460e-b124-397e04da7c01 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:39.905857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51e74d-20e0-460e-b124-397e04da7c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:39.906099 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51e74d-20e0-460e-b124-397e04da7c01 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:39.906789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51e74d-20e0-460e-b124-397e04da7c01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:39.908728 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51e74d-20e0-460e-b124-397e04da7c01 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:40.056608 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 998/2023] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:07:39 2022] POST /identity/v3/users => generated 287 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:07:40.063785 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d9976-c274-4394-bcd6-14a72cc3016a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:40.078807 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d9976-c274-4394-bcd6-14a72cc3016a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:40.084350 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc446eb-c7a6-4ce4-a814-05a3f178988a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:40.085133 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc446eb-c7a6-4ce4-a814-05a3f178988a admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:40.085539 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc446eb-c7a6-4ce4-a814-05a3f178988a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:40.086101 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc446eb-c7a6-4ce4-a814-05a3f178988a admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:40.091096 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc446eb-c7a6-4ce4-a814-05a3f178988a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:40.091096 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc446eb-c7a6-4ce4-a814-05a3f178988a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:40.115767 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1026/2024] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:40.120021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51e74d-20e0-460e-b124-397e04da7c01 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:40.135441 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51e74d-20e0-460e-b124-397e04da7c01 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:40.144603 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c65f6-0a84-47c4-a90e-c7853133198f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:40.146075 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c65f6-0a84-47c4-a90e-c7853133198f admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:40.157748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c65f6-0a84-47c4-a90e-c7853133198f admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:40.157748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c65f6-0a84-47c4-a90e-c7853133198f admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:40.157748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093c65f6-0a84-47c4-a90e-c7853133198f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:40.157748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093c65f6-0a84-47c4-a90e-c7853133198f admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:40.168732 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc446eb-c7a6-4ce4-a814-05a3f178988a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:40.181534 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 999/2025] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:40.183540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc446eb-c7a6-4ce4-a814-05a3f178988a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:40.190463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cda55-887a-4878-b35b-17451b6d2054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:40.191231 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cda55-887a-4878-b35b-17451b6d2054 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:40.191617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cda55-887a-4878-b35b-17451b6d2054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:40.191947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cda55-887a-4878-b35b-17451b6d2054 admin admin] PATH_INFO: `/v3/projects/9d3d987d8b484c639b4c121b63e7657b/users/8e029b3a928940a1bc7e6ac883595069/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:40.210040 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1cda55-887a-4878-b35b-17451b6d2054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d3d987d8b484c639b4c121b63e7657b, user_id=8e029b3a928940a1bc7e6ac883595069, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:40.213901 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1cda55-887a-4878-b35b-17451b6d2054 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:40.371026 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbc1c0-776d-45b5-83ee-3009cc60840f None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:40.371688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbc1c0-776d-45b5-83ee-3009cc60840f None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:40.371688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbc1c0-776d-45b5-83ee-3009cc60840f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:40.376501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1027/2026] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:40 2022] PUT /identity/v3/projects/9d3d987d8b484c639b4c121b63e7657b/users/8e029b3a928940a1bc7e6ac883595069/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:40.408173 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-57fbc1c0-776d-45b5-83ee-3009cc60840f None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:40.409835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-57fbc1c0-776d-45b5-83ee-3009cc60840f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:40.512682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:40.526407 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1000/2027] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:40.536581 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cda55-887a-4878-b35b-17451b6d2054 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:40.539316 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2f1cda55-887a-4878-b35b-17451b6d2054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:40.649368 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cda55-887a-4878-b35b-17451b6d2054 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:40.664969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198f507-3001-498f-9329-8d22871200fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:40.666737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198f507-3001-498f-9329-8d22871200fa service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:40.667429 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198f507-3001-498f-9329-8d22871200fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:40.668127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3198f507-3001-498f-9329-8d22871200fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:40.686357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3198f507-3001-498f-9329-8d22871200fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:40.693144 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3198f507-3001-498f-9329-8d22871200fa service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:40.706715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1028/2028] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:07:40.970395 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:41.825521 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:42.088761 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:42.669210 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:42.838289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7ca2b-3801-40e1-9590-26401b63990e None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:42.839174 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7ca2b-3801-40e1-9590-26401b63990e None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:42.839174 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7ca2b-3801-40e1-9590-26401b63990e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:42.905816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-9ed7ca2b-3801-40e1-9590-26401b63990e None None] MFA Rules not processed for user `8e029b3a928940a1bc7e6ac883595069`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:42.913810 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-9ed7ca2b-3801-40e1-9590-26401b63990e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:42.942466 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:43.065144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1001/2029] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:43.066805 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198f507-3001-498f-9329-8d22871200fa service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:43.094384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3198f507-3001-498f-9329-8d22871200fa service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:43.107097 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:43.122040 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceae996-09e4-4b10-b5f0-ece75767dcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:43.122783 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceae996-09e4-4b10-b5f0-ece75767dcd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:43.123989 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceae996-09e4-4b10-b5f0-ece75767dcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:43.124344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceae996-09e4-4b10-b5f0-ece75767dcd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:43.125114 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceae996-09e4-4b10-b5f0-ece75767dcd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:43.127568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceae996-09e4-4b10-b5f0-ece75767dcd4 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:43.249767 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:43.257372 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1029/2030] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:07:43 2022] POST /identity/v3/projects => generated 371 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:43.258650 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7ca2b-3801-40e1-9590-26401b63990e None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:43.280292 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7ca2b-3801-40e1-9590-26401b63990e None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:43.397125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:43.514198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d138f-19ad-475b-bbc6-2017abc2312c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:43.514198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d138f-19ad-475b-bbc6-2017abc2312c admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:43.514198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d138f-19ad-475b-bbc6-2017abc2312c admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:43.514198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d138f-19ad-475b-bbc6-2017abc2312c admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:43.514573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d138f-19ad-475b-bbc6-2017abc2312c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:43.515587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5d138f-19ad-475b-bbc6-2017abc2312c admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:43.547282 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:43.720224 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1002/2031] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:07:43 2022] POST /identity/v3/users => generated 292 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:43.725245 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceae996-09e4-4b10-b5f0-ece75767dcd4 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:43.753098 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceae996-09e4-4b10-b5f0-ece75767dcd4 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:43.771846 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe03fb-ad5d-4204-871d-9e668d5ac82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:43.773277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe03fb-ad5d-4204-871d-9e668d5ac82b admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:43.773277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe03fb-ad5d-4204-871d-9e668d5ac82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:43.773277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe03fb-ad5d-4204-871d-9e668d5ac82b admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:43.774427 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe03fb-ad5d-4204-871d-9e668d5ac82b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:43.777462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efe03fb-ad5d-4204-871d-9e668d5ac82b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:43.812761 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1030/2032] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:43.814184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d138f-19ad-475b-bbc6-2017abc2312c admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:43.857676 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d138f-19ad-475b-bbc6-2017abc2312c admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:43.868478 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bb46c-adef-4f17-be09-db8b4b7df3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:43.869180 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bb46c-adef-4f17-be09-db8b4b7df3a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:43.869464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bb46c-adef-4f17-be09-db8b4b7df3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:43.869743 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966bb46c-adef-4f17-be09-db8b4b7df3a2 admin admin] PATH_INFO: `/v3/projects/6648f2058e6b4b93943cf95756c9a391/users/e59092ffb4374aaa8800069c8b1fdf18/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:43.891778 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bb46c-adef-4f17-be09-db8b4b7df3a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6648f2058e6b4b93943cf95756c9a391, user_id=e59092ffb4374aaa8800069c8b1fdf18, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:43.899878 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966bb46c-adef-4f17-be09-db8b4b7df3a2 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:43.908156 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:43.983732 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1003/2033] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:43 2022] PUT /identity/v3/projects/6648f2058e6b4b93943cf95756c9a391/users/e59092ffb4374aaa8800069c8b1fdf18/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:43.988517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe03fb-ad5d-4204-871d-9e668d5ac82b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:44.006137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efe03fb-ad5d-4204-871d-9e668d5ac82b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:44.063221 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:44.114543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c83e80-0087-4f94-b7ff-48f19d51e79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:44.116806 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c83e80-0087-4f94-b7ff-48f19d51e79c admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:44.117264 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c83e80-0087-4f94-b7ff-48f19d51e79c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:44.117856 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c83e80-0087-4f94-b7ff-48f19d51e79c admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:44.119099 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c83e80-0087-4f94-b7ff-48f19d51e79c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:44.122406 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c83e80-0087-4f94-b7ff-48f19d51e79c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:44.136536 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1031/2034] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:44.140923 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bb46c-adef-4f17-be09-db8b4b7df3a2 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:44.164229 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966bb46c-adef-4f17-be09-db8b4b7df3a2 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:44.175208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62937cb-0373-425e-8933-9da1ed221935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:44.176849 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62937cb-0373-425e-8933-9da1ed221935 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:44.177469 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62937cb-0373-425e-8933-9da1ed221935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:44.177758 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62937cb-0373-425e-8933-9da1ed221935 admin admin] PATH_INFO: `/v3/projects/6648f2058e6b4b93943cf95756c9a391/users/e59092ffb4374aaa8800069c8b1fdf18/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:44.180181 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62937cb-0373-425e-8933-9da1ed221935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6648f2058e6b4b93943cf95756c9a391, user_id=e59092ffb4374aaa8800069c8b1fdf18, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:44.183811 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62937cb-0373-425e-8933-9da1ed221935 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:44.202990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1004/2035] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:44 2022] PUT /identity/v3/projects/6648f2058e6b4b93943cf95756c9a391/users/e59092ffb4374aaa8800069c8b1fdf18/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:07:45.125528 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c83e80-0087-4f94-b7ff-48f19d51e79c admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:45.127691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-91c83e80-0087-4f94-b7ff-48f19d51e79c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:45.144446 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c83e80-0087-4f94-b7ff-48f19d51e79c admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:45.259220 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af094466-67ab-4e9b-861d-302d8fa7d563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:45.259845 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af094466-67ab-4e9b-861d-302d8fa7d563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:45.260136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af094466-67ab-4e9b-861d-302d8fa7d563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:45.260421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af094466-67ab-4e9b-861d-302d8fa7d563 admin admin] PATH_INFO: `/v3/users/64e98a0d50994e19bb2706545dd843d4` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:45.262368 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af094466-67ab-4e9b-861d-302d8fa7d563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64e98a0d50994e19bb2706545dd843d4)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:45.264504 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af094466-67ab-4e9b-861d-302d8fa7d563 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:45.537652 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-af094466-67ab-4e9b-861d-302d8fa7d563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64e98a0d50994e19bb2706545dd843d4'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:45.554079 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-af094466-67ab-4e9b-861d-302d8fa7d563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64e98a0d50994e19bb2706545dd843d4'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:45.572166 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-af094466-67ab-4e9b-861d-302d8fa7d563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64e98a0d50994e19bb2706545dd843d4'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:45.630668 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-af094466-67ab-4e9b-861d-302d8fa7d563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64e98a0d50994e19bb2706545dd843d4'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:45.631767 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-af094466-67ab-4e9b-861d-302d8fa7d563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64e98a0d50994e19bb2706545dd843d4'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:45.740278 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1032/2036] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:07:45 2022] DELETE /identity/v3/users/64e98a0d50994e19bb2706545dd843d4 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:45.745614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235eda5e-e160-4129-8da3-87cf9221ada3 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:45.746179 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235eda5e-e160-4129-8da3-87cf9221ada3 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:45.746179 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235eda5e-e160-4129-8da3-87cf9221ada3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:45.800309 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-235eda5e-e160-4129-8da3-87cf9221ada3 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:45.806625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-235eda5e-e160-4129-8da3-87cf9221ada3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:45.960826 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1005/2037] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:07:45.968901 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af094466-67ab-4e9b-861d-302d8fa7d563 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:45.970510 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-af094466-67ab-4e9b-861d-302d8fa7d563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:46.063211 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:46.073570 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af094466-67ab-4e9b-861d-302d8fa7d563 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:46.077911 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357981f2-57a7-4cc7-942f-58e900104b45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:46.078710 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357981f2-57a7-4cc7-942f-58e900104b45 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:46.079261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357981f2-57a7-4cc7-942f-58e900104b45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:46.079653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357981f2-57a7-4cc7-942f-58e900104b45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:46.091977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357981f2-57a7-4cc7-942f-58e900104b45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:46.094539 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357981f2-57a7-4cc7-942f-58e900104b45 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:46.106720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1033/2038] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:07:46.549851 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1611dd1-9f31-460b-b804-b1a155f53a3d None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:46.550401 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1611dd1-9f31-460b-b804-b1a155f53a3d None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:46.550995 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1611dd1-9f31-460b-b804-b1a155f53a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:46.620361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-f1611dd1-9f31-460b-b804-b1a155f53a3d None None] MFA Rules not processed for user `e59092ffb4374aaa8800069c8b1fdf18`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:46.621642 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f1611dd1-9f31-460b-b804-b1a155f53a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:46.630180 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357981f2-57a7-4cc7-942f-58e900104b45 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:46.632666 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-357981f2-57a7-4cc7-942f-58e900104b45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:46.656907 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357981f2-57a7-4cc7-942f-58e900104b45 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:46.659199 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d0a70-a0bf-42d4-abf9-b6a6645fa8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:46.660759 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d0a70-a0bf-42d4-abf9-b6a6645fa8b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:46.661185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d0a70-a0bf-42d4-abf9-b6a6645fa8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:46.661556 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72d0a70-a0bf-42d4-abf9-b6a6645fa8b1 admin admin] PATH_INFO: `/v3/projects/bb3948173f294a9381136b4509652fc7` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:46.663410 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d0a70-a0bf-42d4-abf9-b6a6645fa8b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb3948173f294a9381136b4509652fc7)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:46.666671 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72d0a70-a0bf-42d4-abf9-b6a6645fa8b1 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:46.782268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1006/2039] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:46 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:46.807829 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1611dd1-9f31-460b-b804-b1a155f53a3d None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:46.811441 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f1611dd1-9f31-460b-b804-b1a155f53a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:46.839724 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b72d0a70-a0bf-42d4-abf9-b6a6645fa8b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb3948173f294a9381136b4509652fc7', 'request_id': 'req-b72d0a70-a0bf-42d4-abf9-b6a6645fa8b1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:46.847170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b72d0a70-a0bf-42d4-abf9-b6a6645fa8b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb3948173f294a9381136b4509652fc7', 'request_id': 'req-b72d0a70-a0bf-42d4-abf9-b6a6645fa8b1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:46.899030 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1611dd1-9f31-460b-b804-b1a155f53a3d None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:46.905645 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:46.933344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b72d0a70-a0bf-42d4-abf9-b6a6645fa8b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb3948173f294a9381136b4509652fc7', 'request_id': 'req-b72d0a70-a0bf-42d4-abf9-b6a6645fa8b1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:46.963822 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b72d0a70-a0bf-42d4-abf9-b6a6645fa8b1 admin admin] The token cache is being invalidate because project bb3948173f294a9381136b4509652fc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:07:46.966932 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b72d0a70-a0bf-42d4-abf9-b6a6645fa8b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:46.966932 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b72d0a70-a0bf-42d4-abf9-b6a6645fa8b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:46.970595 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1034/2040] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:07:46 2022] DELETE /identity/v3/projects/bb3948173f294a9381136b4509652fc7 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:46.971600 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d0a70-a0bf-42d4-abf9-b6a6645fa8b1 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:46.977715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b72d0a70-a0bf-42d4-abf9-b6a6645fa8b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:46.985006 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540f7fc-5a2e-4df2-9fef-048a9ebd8871 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:46.985664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540f7fc-5a2e-4df2-9fef-048a9ebd8871 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:46.986002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540f7fc-5a2e-4df2-9fef-048a9ebd8871 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:46.986282 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540f7fc-5a2e-4df2-9fef-048a9ebd8871 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:46.991013 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-2540f7fc-5a2e-4df2-9fef-048a9ebd8871 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:47.107657 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72d0a70-a0bf-42d4-abf9-b6a6645fa8b1 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:47.109175 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2540f7fc-5a2e-4df2-9fef-048a9ebd8871 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:47.110680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2540f7fc-5a2e-4df2-9fef-048a9ebd8871 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:47.124396 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132f330-6e7a-47f1-a183-a461ebc50c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:47.125047 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1007/2041] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:07:47.125191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132f330-6e7a-47f1-a183-a461ebc50c21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:47.125589 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132f330-6e7a-47f1-a183-a461ebc50c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:47.125939 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132f330-6e7a-47f1-a183-a461ebc50c21 admin admin] PATH_INFO: `/v3/users/3dd7c185a642400c99ec999eacf859bb` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:47.132931 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9132f330-6e7a-47f1-a183-a461ebc50c21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dd7c185a642400c99ec999eacf859bb)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:47.132931 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9132f330-6e7a-47f1-a183-a461ebc50c21 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:47.377496 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9132f330-6e7a-47f1-a183-a461ebc50c21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dd7c185a642400c99ec999eacf859bb'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:47.405688 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9132f330-6e7a-47f1-a183-a461ebc50c21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dd7c185a642400c99ec999eacf859bb'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:47.406621 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9132f330-6e7a-47f1-a183-a461ebc50c21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dd7c185a642400c99ec999eacf859bb'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:47.440424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9132f330-6e7a-47f1-a183-a461ebc50c21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dd7c185a642400c99ec999eacf859bb'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:47.441263 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9132f330-6e7a-47f1-a183-a461ebc50c21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dd7c185a642400c99ec999eacf859bb'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:47.443616 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:47.573216 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1035/2042] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:07:46 2022] DELETE /identity/v3/users/3dd7c185a642400c99ec999eacf859bb => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:48.149542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540f7fc-5a2e-4df2-9fef-048a9ebd8871 service cinder] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:48.167814 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540f7fc-5a2e-4df2-9fef-048a9ebd8871 service cinder] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:48.284882 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef994fb-3669-4747-a82d-87a1ac29978a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:48.285610 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef994fb-3669-4747-a82d-87a1ac29978a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:48.285936 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef994fb-3669-4747-a82d-87a1ac29978a admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:48.286217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef994fb-3669-4747-a82d-87a1ac29978a admin admin] PATH_INFO: `/v3/projects/de1cd0d023d143d8b46fc9648aacaabc` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:48.287522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef994fb-3669-4747-a82d-87a1ac29978a admin admin] RBAC: Authorizing `identity:delete_project(project_id=de1cd0d023d143d8b46fc9648aacaabc)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:48.289558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef994fb-3669-4747-a82d-87a1ac29978a admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:48.497494 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8ef994fb-3669-4747-a82d-87a1ac29978a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de1cd0d023d143d8b46fc9648aacaabc', 'request_id': 'req-8ef994fb-3669-4747-a82d-87a1ac29978a'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:48.504780 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8ef994fb-3669-4747-a82d-87a1ac29978a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de1cd0d023d143d8b46fc9648aacaabc', 'request_id': 'req-8ef994fb-3669-4747-a82d-87a1ac29978a'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:48.512169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8ef994fb-3669-4747-a82d-87a1ac29978a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de1cd0d023d143d8b46fc9648aacaabc', 'request_id': 'req-8ef994fb-3669-4747-a82d-87a1ac29978a'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:48.515491 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:49.073954 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8ef994fb-3669-4747-a82d-87a1ac29978a admin admin] The token cache is being invalidate because project de1cd0d023d143d8b46fc9648aacaabc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:07:49.077779 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8ef994fb-3669-4747-a82d-87a1ac29978a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:49.077779 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8ef994fb-3669-4747-a82d-87a1ac29978a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:49.077779 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1008/2043] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:07:48 2022] DELETE /identity/v3/projects/de1cd0d023d143d8b46fc9648aacaabc => generated 0 bytes in 928 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:49.082227 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea590a-6210-494c-8e12-6f7b72819c63 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:49.082723 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea590a-6210-494c-8e12-6f7b72819c63 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:49.082816 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ea590a-6210-494c-8e12-6f7b72819c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:49.125945 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-99ea590a-6210-494c-8e12-6f7b72819c63 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:49.127186 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-99ea590a-6210-494c-8e12-6f7b72819c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:49.209940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1036/2044] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:49.213853 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dfd9e0-9ca6-4794-923f-78ec102ff640 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:49.214337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dfd9e0-9ca6-4794-923f-78ec102ff640 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:49.214555 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dfd9e0-9ca6-4794-923f-78ec102ff640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:49.239542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-97dfd9e0-9ca6-4794-923f-78ec102ff640 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:49.240793 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-97dfd9e0-9ca6-4794-923f-78ec102ff640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:49.257075 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1009/2045] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:49.258090 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea590a-6210-494c-8e12-6f7b72819c63 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:49.271096 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ea590a-6210-494c-8e12-6f7b72819c63 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:49.283062 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0990e-3da9-44d2-a12a-47f544a20eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:49.293947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0990e-3da9-44d2-a12a-47f544a20eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:49.293947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0990e-3da9-44d2-a12a-47f544a20eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:49.293947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f0990e-3da9-44d2-a12a-47f544a20eb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:49.293947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0990e-3da9-44d2-a12a-47f544a20eb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:07:49.293947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0990e-3da9-44d2-a12a-47f544a20eb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:49.293947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f0990e-3da9-44d2-a12a-47f544a20eb5 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:49.306579 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1037/2046] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:07:49.311118 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dfd9e0-9ca6-4794-923f-78ec102ff640 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:49.323036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dfd9e0-9ca6-4794-923f-78ec102ff640 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:49.334459 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeba960-acd6-4f89-8649-b555b1e24b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:49.334459 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeba960-acd6-4f89-8649-b555b1e24b14 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:49.334459 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeba960-acd6-4f89-8649-b555b1e24b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:49.334459 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeba960-acd6-4f89-8649-b555b1e24b14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:49.334801 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeba960-acd6-4f89-8649-b555b1e24b14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:49.335151 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeba960-acd6-4f89-8649-b555b1e24b14 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:49.471695 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1010/2047] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:07:49 2022] POST /identity/v3/projects => generated 369 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:07:49.480188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0990e-3da9-44d2-a12a-47f544a20eb5 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:49.490494 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f0990e-3da9-44d2-a12a-47f544a20eb5 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:49.603327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9cc562-f75f-4f61-a50e-af04ebb53733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:49.603974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cc562-f75f-4f61-a50e-af04ebb53733 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:49.604254 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cc562-f75f-4f61-a50e-af04ebb53733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:49.604526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9cc562-f75f-4f61-a50e-af04ebb53733 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:49.610979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9cc562-f75f-4f61-a50e-af04ebb53733 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:49.613189 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9cc562-f75f-4f61-a50e-af04ebb53733 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:49.722111 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1038/2048] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:07:49 2022] POST /identity/v3/users => generated 285 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:49.726298 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeba960-acd6-4f89-8649-b555b1e24b14 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:49.739711 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeba960-acd6-4f89-8649-b555b1e24b14 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:49.745628 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe874f-e3eb-4af9-8078-26d1642c1ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:49.751302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe874f-e3eb-4af9-8078-26d1642c1ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:49.751302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe874f-e3eb-4af9-8078-26d1642c1ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:49.751302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe874f-e3eb-4af9-8078-26d1642c1ca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:49.751302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe874f-e3eb-4af9-8078-26d1642c1ca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:49.751302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe874f-e3eb-4af9-8078-26d1642c1ca0 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:49.762742 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1011/2049] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:07:49.767520 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9cc562-f75f-4f61-a50e-af04ebb53733 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:49.784612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9cc562-f75f-4f61-a50e-af04ebb53733 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:49.790273 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d052e-54db-492b-88e7-cd1e8fa15e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:49.790955 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d052e-54db-492b-88e7-cd1e8fa15e94 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:49.791246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d052e-54db-492b-88e7-cd1e8fa15e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:49.791742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d052e-54db-492b-88e7-cd1e8fa15e94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:49.792438 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d052e-54db-492b-88e7-cd1e8fa15e94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:49.798887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d052e-54db-492b-88e7-cd1e8fa15e94 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:49.812254 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1039/2050] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:49.816273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe874f-e3eb-4af9-8078-26d1642c1ca0 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:49.827459 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe874f-e3eb-4af9-8078-26d1642c1ca0 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:49.854300 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfa465-e6b1-4815-b7e4-da96fb7fe671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:49.855614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfa465-e6b1-4815-b7e4-da96fb7fe671 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:49.855913 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfa465-e6b1-4815-b7e4-da96fb7fe671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:49.856199 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfa465-e6b1-4815-b7e4-da96fb7fe671 admin admin] PATH_INFO: `/v3/projects/26323ee7d3014b80986cea144ca68b1c/users/6d9f61a53ca64d59a1b24420320472b2/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:49.878386 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfa465-e6b1-4815-b7e4-da96fb7fe671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26323ee7d3014b80986cea144ca68b1c, user_id=6d9f61a53ca64d59a1b24420320472b2, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:49.882220 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfa465-e6b1-4815-b7e4-da96fb7fe671 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:49.958512 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1012/2051] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:49 2022] PUT /identity/v3/projects/26323ee7d3014b80986cea144ca68b1c/users/6d9f61a53ca64d59a1b24420320472b2/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:07:49.971216 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7dceb-2715-4a4e-97b0-bf98f4c19716 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:49.971845 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7dceb-2715-4a4e-97b0-bf98f4c19716 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:49.972263 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d7dceb-2715-4a4e-97b0-bf98f4c19716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:50.009837 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-45d7dceb-2715-4a4e-97b0-bf98f4c19716 None None] MFA Rules not processed for user `6d9f61a53ca64d59a1b24420320472b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:50.011125 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-45d7dceb-2715-4a4e-97b0-bf98f4c19716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:50.074537 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:50.124741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1040/2052] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:50.126491 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfa465-e6b1-4815-b7e4-da96fb7fe671 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:50.232492 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfa465-e6b1-4815-b7e4-da96fb7fe671 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:50.238036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d9b7b-f722-46f9-99d6-1a9a8f896f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:50.240471 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d9b7b-f722-46f9-99d6-1a9a8f896f3b admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:50.240471 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d9b7b-f722-46f9-99d6-1a9a8f896f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:50.240471 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d9b7b-f722-46f9-99d6-1a9a8f896f3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:50.240471 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d9b7b-f722-46f9-99d6-1a9a8f896f3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:50.244004 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d9b7b-f722-46f9-99d6-1a9a8f896f3b admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:50.277770 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d7dceb-2715-4a4e-97b0-bf98f4c19716 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:50.280980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-45d7dceb-2715-4a4e-97b0-bf98f4c19716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:50.333796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1013/2053] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:07:50 2022] POST /identity/v3/projects => generated 369 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:07:50.338867 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d9b7b-f722-46f9-99d6-1a9a8f896f3b admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:50.392004 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d7dceb-2715-4a4e-97b0-bf98f4c19716 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:50.471184 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d9b7b-f722-46f9-99d6-1a9a8f896f3b admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:50.500939 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e4a9e-1351-4b7d-875b-f20a2ba447ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:50.501690 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e4a9e-1351-4b7d-875b-f20a2ba447ce admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:50.502009 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e4a9e-1351-4b7d-875b-f20a2ba447ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:50.502287 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8e4a9e-1351-4b7d-875b-f20a2ba447ce admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:50.502979 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8e4a9e-1351-4b7d-875b-f20a2ba447ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:50.504935 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8e4a9e-1351-4b7d-875b-f20a2ba447ce admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:50.531340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8edda50-e776-4984-99fc-9d392a8b529e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:50.531600 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8edda50-e776-4984-99fc-9d392a8b529e service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:50.531896 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8edda50-e776-4984-99fc-9d392a8b529e service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:50.532181 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8edda50-e776-4984-99fc-9d392a8b529e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:50.543582 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b8edda50-e776-4984-99fc-9d392a8b529e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:50.660893 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1014/2054] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:07:50 2022] POST /identity/v3/users => generated 291 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:07:50.662180 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e4a9e-1351-4b7d-875b-f20a2ba447ce admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:50.667404 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8edda50-e776-4984-99fc-9d392a8b529e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:50.668554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-3f8e4a9e-1351-4b7d-875b-f20a2ba447ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:50.668952 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8edda50-e776-4984-99fc-9d392a8b529e service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:50.678578 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:DELETE=1,keystone:SELECT=93,keystone:INSERT=6 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:50.683226 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8e4a9e-1351-4b7d-875b-f20a2ba447ce admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:50.683669 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1041/2055] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:07:50.685247 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8edda50-e776-4984-99fc-9d392a8b529e service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:50.699289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8edda50-e776-4984-99fc-9d392a8b529e service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:50.717664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2720206a-1b19-48a7-8b8c-7063aecbdbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:50.720828 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2720206a-1b19-48a7-8b8c-7063aecbdbd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:50.721128 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2720206a-1b19-48a7-8b8c-7063aecbdbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:50.721414 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2720206a-1b19-48a7-8b8c-7063aecbdbd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:50.722436 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2720206a-1b19-48a7-8b8c-7063aecbdbd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:50.724281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2720206a-1b19-48a7-8b8c-7063aecbdbd1 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:50.732397 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65136bf2-a548-44cf-8bf6-5a31a1ea0ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:50.733339 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65136bf2-a548-44cf-8bf6-5a31a1ea0ba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:50.734552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65136bf2-a548-44cf-8bf6-5a31a1ea0ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:50.735075 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65136bf2-a548-44cf-8bf6-5a31a1ea0ba4 admin admin] PATH_INFO: `/v3/users/31c682d0477d4be3b092a7fced6ce4d1` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:50.742825 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65136bf2-a548-44cf-8bf6-5a31a1ea0ba4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31c682d0477d4be3b092a7fced6ce4d1)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:50.749431 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65136bf2-a548-44cf-8bf6-5a31a1ea0ba4 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:50.749615 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1042/2056] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:50.756124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2720206a-1b19-48a7-8b8c-7063aecbdbd1 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:50.763310 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2720206a-1b19-48a7-8b8c-7063aecbdbd1 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:50.771408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e4792-c240-4562-a935-fc2847c8749d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:50.773117 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e4792-c240-4562-a935-fc2847c8749d admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:50.773535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e4792-c240-4562-a935-fc2847c8749d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:50.773666 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01e4792-c240-4562-a935-fc2847c8749d admin admin] PATH_INFO: `/v3/projects/6343e71c05454ea49b14d2d84d12c62b/users/1686f22c057c4727b8e9097ae9bb4aed/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:50.799465 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e4792-c240-4562-a935-fc2847c8749d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6343e71c05454ea49b14d2d84d12c62b, user_id=1686f22c057c4727b8e9097ae9bb4aed, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:50.804289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01e4792-c240-4562-a935-fc2847c8749d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:50.877832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1043/2057] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:50 2022] PUT /identity/v3/projects/6343e71c05454ea49b14d2d84d12c62b/users/1686f22c057c4727b8e9097ae9bb4aed/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:50.882465 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e4792-c240-4562-a935-fc2847c8749d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:50.943832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:51.020167 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01e4792-c240-4562-a935-fc2847c8749d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:51.027593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c927b0-e0e8-42c9-b0d2-418794ec741d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:51.028543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c927b0-e0e8-42c9-b0d2-418794ec741d admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:51.028983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c927b0-e0e8-42c9-b0d2-418794ec741d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:51.029435 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c927b0-e0e8-42c9-b0d2-418794ec741d admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:51.030400 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c927b0-e0e8-42c9-b0d2-418794ec741d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:51.032387 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c927b0-e0e8-42c9-b0d2-418794ec741d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:51.047190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1044/2058] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:50 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:51.051159 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c927b0-e0e8-42c9-b0d2-418794ec741d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:51.064586 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c927b0-e0e8-42c9-b0d2-418794ec741d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:51.069625 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffd3a6-9f68-43fb-87cc-a000c02deb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:51.072092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffd3a6-9f68-43fb-87cc-a000c02deb77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:51.072274 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffd3a6-9f68-43fb-87cc-a000c02deb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:51.073215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffd3a6-9f68-43fb-87cc-a000c02deb77 admin admin] PATH_INFO: `/v3/projects/6343e71c05454ea49b14d2d84d12c62b/users/1686f22c057c4727b8e9097ae9bb4aed/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:51.076121 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffd3a6-9f68-43fb-87cc-a000c02deb77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6343e71c05454ea49b14d2d84d12c62b, user_id=1686f22c057c4727b8e9097ae9bb4aed, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:51.079054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cffd3a6-9f68-43fb-87cc-a000c02deb77 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:51.083773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-65136bf2-a548-44cf-8bf6-5a31a1ea0ba4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31c682d0477d4be3b092a7fced6ce4d1'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:51.093065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-65136bf2-a548-44cf-8bf6-5a31a1ea0ba4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31c682d0477d4be3b092a7fced6ce4d1'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:51.094026 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-65136bf2-a548-44cf-8bf6-5a31a1ea0ba4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31c682d0477d4be3b092a7fced6ce4d1'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:51.097660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1045/2059] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:51 2022] PUT /identity/v3/projects/6343e71c05454ea49b14d2d84d12c62b/users/1686f22c057c4727b8e9097ae9bb4aed/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:07:51.108021 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03dbf6b-f1a9-434a-a064-44918f5c007d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:51.108517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03dbf6b-f1a9-434a-a064-44918f5c007d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:51.108875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03dbf6b-f1a9-434a-a064-44918f5c007d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:51.147685 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-e03dbf6b-f1a9-434a-a064-44918f5c007d None None] MFA Rules not processed for user `1686f22c057c4727b8e9097ae9bb4aed`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:51.149136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e03dbf6b-f1a9-434a-a064-44918f5c007d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:51.204145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-65136bf2-a548-44cf-8bf6-5a31a1ea0ba4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31c682d0477d4be3b092a7fced6ce4d1'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:51.225332 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-65136bf2-a548-44cf-8bf6-5a31a1ea0ba4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31c682d0477d4be3b092a7fced6ce4d1'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:51.233566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1015/2060] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:07:50 2022] DELETE /identity/v3/users/31c682d0477d4be3b092a7fced6ce4d1 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:51.238385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deec6c2-d395-40d6-9816-9f25b5016fe3 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:51.238737 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deec6c2-d395-40d6-9816-9f25b5016fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:51.239018 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deec6c2-d395-40d6-9816-9f25b5016fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:51.262363 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1046/2061] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:51.272456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-2deec6c2-d395-40d6-9816-9f25b5016fe3 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:51.275899 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03dbf6b-f1a9-434a-a064-44918f5c007d None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:51.276016 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-2deec6c2-d395-40d6-9816-9f25b5016fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:51.278785 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e03dbf6b-f1a9-434a-a064-44918f5c007d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:51.357920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1016/2062] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:51.367735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deec6c2-d395-40d6-9816-9f25b5016fe3 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:51.369978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-2deec6c2-d395-40d6-9816-9f25b5016fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:51.384877 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03dbf6b-f1a9-434a-a064-44918f5c007d None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:51.442339 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:51.457607 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50425565-70bd-4094-b070-0e56aa20c3c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:51.458281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50425565-70bd-4094-b070-0e56aa20c3c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:51.458569 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50425565-70bd-4094-b070-0e56aa20c3c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:51.458851 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50425565-70bd-4094-b070-0e56aa20c3c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:51.474974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deec6c2-d395-40d6-9816-9f25b5016fe3 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:51.483911 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d55cc5-c7e8-4b1b-b458-a5d5b876b215 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:51.484571 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d55cc5-c7e8-4b1b-b458-a5d5b876b215 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:51.484855 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d55cc5-c7e8-4b1b-b458-a5d5b876b215 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:51.485475 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d55cc5-c7e8-4b1b-b458-a5d5b876b215 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:51.499185 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d55cc5-c7e8-4b1b-b458-a5d5b876b215 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:51.500846 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d55cc5-c7e8-4b1b-b458-a5d5b876b215 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:51.533700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1017/2063] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:07:51.556444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50425565-70bd-4094-b070-0e56aa20c3c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:51.559127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50425565-70bd-4094-b070-0e56aa20c3c5 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:51.608787 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1047/2064] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:07:52.112437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d55cc5-c7e8-4b1b-b458-a5d5b876b215 service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:52.112703 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-b6d55cc5-c7e8-4b1b-b458-a5d5b876b215 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:52.130883 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d55cc5-c7e8-4b1b-b458-a5d5b876b215 service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:52.137562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b549b-40a1-4ce9-ab62-51830a2253b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:52.138402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b549b-40a1-4ce9-ab62-51830a2253b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:52.138782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b549b-40a1-4ce9-ab62-51830a2253b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:52.139216 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b549b-40a1-4ce9-ab62-51830a2253b1 admin admin] PATH_INFO: `/v3/projects/3960392fe03e4cc995dd6fc7841382bc` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:52.159591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b549b-40a1-4ce9-ab62-51830a2253b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3960392fe03e4cc995dd6fc7841382bc)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:52.163065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b549b-40a1-4ce9-ab62-51830a2253b1 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:52.206680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:52.366940 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-cb6b549b-40a1-4ce9-ab62-51830a2253b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3960392fe03e4cc995dd6fc7841382bc', 'request_id': 'req-cb6b549b-40a1-4ce9-ab62-51830a2253b1'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:52.376530 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-cb6b549b-40a1-4ce9-ab62-51830a2253b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3960392fe03e4cc995dd6fc7841382bc', 'request_id': 'req-cb6b549b-40a1-4ce9-ab62-51830a2253b1'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:52.403948 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-cb6b549b-40a1-4ce9-ab62-51830a2253b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3960392fe03e4cc995dd6fc7841382bc', 'request_id': 'req-cb6b549b-40a1-4ce9-ab62-51830a2253b1'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:52.519607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-cb6b549b-40a1-4ce9-ab62-51830a2253b1 admin admin] The token cache is being invalidate because project 3960392fe03e4cc995dd6fc7841382bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:07:52.531403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-cb6b549b-40a1-4ce9-ab62-51830a2253b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:52.535321 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-cb6b549b-40a1-4ce9-ab62-51830a2253b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:52.541404 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1018/2065] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:07:52 2022] DELETE /identity/v3/projects/3960392fe03e4cc995dd6fc7841382bc => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:52.546948 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69b7ea-c959-407b-954c-370a3c31e716 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:52.547791 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69b7ea-c959-407b-954c-370a3c31e716 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:52.548255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69b7ea-c959-407b-954c-370a3c31e716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:52.594209 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5e69b7ea-c959-407b-954c-370a3c31e716 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:52.595347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5e69b7ea-c959-407b-954c-370a3c31e716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:52.760171 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1048/2066] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:52.769714 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453e6c6-9b48-41a2-995a-f18d952f53d8 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:52.770442 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453e6c6-9b48-41a2-995a-f18d952f53d8 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:52.770926 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2453e6c6-9b48-41a2-995a-f18d952f53d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:52.815312 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-2453e6c6-9b48-41a2-995a-f18d952f53d8 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:52.816482 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-2453e6c6-9b48-41a2-995a-f18d952f53d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:52.825779 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1019/2067] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:07:52.830961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69b7ea-c959-407b-954c-370a3c31e716 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:52.850050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e69b7ea-c959-407b-954c-370a3c31e716 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:52.856351 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5530c-be3a-438a-87eb-795ab24cc7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:52.857748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5530c-be3a-438a-87eb-795ab24cc7e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:52.858468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5530c-be3a-438a-87eb-795ab24cc7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:52.858855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de5530c-be3a-438a-87eb-795ab24cc7e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:52.859718 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5530c-be3a-438a-87eb-795ab24cc7e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:07:52.860134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5530c-be3a-438a-87eb-795ab24cc7e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:52.862079 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de5530c-be3a-438a-87eb-795ab24cc7e4 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:52.878599 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1049/2068] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:07:52.882325 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453e6c6-9b48-41a2-995a-f18d952f53d8 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:52.893225 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2453e6c6-9b48-41a2-995a-f18d952f53d8 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:52.898800 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85122bc1-4918-4547-8cfb-6b3f404e26f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:52.899566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85122bc1-4918-4547-8cfb-6b3f404e26f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:52.899852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85122bc1-4918-4547-8cfb-6b3f404e26f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:52.900129 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85122bc1-4918-4547-8cfb-6b3f404e26f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:52.900823 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85122bc1-4918-4547-8cfb-6b3f404e26f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:52.902818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85122bc1-4918-4547-8cfb-6b3f404e26f8 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:53.505255 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1020/2069] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:07:52 2022] POST /identity/v3/projects => generated 383 bytes in 624 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:07:53.510235 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5530c-be3a-438a-87eb-795ab24cc7e4 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:53.520542 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de5530c-be3a-438a-87eb-795ab24cc7e4 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:53.598086 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8bec8-a68b-460f-ad13-7a6821dfc6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:53.599978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8bec8-a68b-460f-ad13-7a6821dfc6df admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:53.600476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8bec8-a68b-460f-ad13-7a6821dfc6df admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:53.601170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8bec8-a68b-460f-ad13-7a6821dfc6df admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:53.602974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c8bec8-a68b-460f-ad13-7a6821dfc6df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:53.605975 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c8bec8-a68b-460f-ad13-7a6821dfc6df admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:53.735641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1050/2070] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:07:53 2022] POST /identity/v3/users => generated 292 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:07:53.740130 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85122bc1-4918-4547-8cfb-6b3f404e26f8 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:53.757490 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85122bc1-4918-4547-8cfb-6b3f404e26f8 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:53.762868 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e0c8c-82ff-40f0-b17d-bb2b66805de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:53.764341 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e0c8c-82ff-40f0-b17d-bb2b66805de3 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:53.764739 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e0c8c-82ff-40f0-b17d-bb2b66805de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:53.765118 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e0c8c-82ff-40f0-b17d-bb2b66805de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:53.766101 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e0c8c-82ff-40f0-b17d-bb2b66805de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:53.768073 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e0c8c-82ff-40f0-b17d-bb2b66805de3 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:53.797697 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1021/2071] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:53.800319 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8bec8-a68b-460f-ad13-7a6821dfc6df admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:53.831479 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8bec8-a68b-460f-ad13-7a6821dfc6df admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:53.856054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33b414e-80a1-4c2a-a119-17279f805794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:53.860816 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b414e-80a1-4c2a-a119-17279f805794 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:53.861348 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b414e-80a1-4c2a-a119-17279f805794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:53.866718 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33b414e-80a1-4c2a-a119-17279f805794 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:53.870401 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b414e-80a1-4c2a-a119-17279f805794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:53.873748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33b414e-80a1-4c2a-a119-17279f805794 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:53.881266 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:53.908760 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e0c8c-82ff-40f0-b17d-bb2b66805de3 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:53.908966 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1051/2072] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:07:53.926071 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e0c8c-82ff-40f0-b17d-bb2b66805de3 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:53.931922 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedfa79-059a-4060-98be-bf5141bd3a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:53.932785 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedfa79-059a-4060-98be-bf5141bd3a69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:53.933147 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedfa79-059a-4060-98be-bf5141bd3a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:53.933506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedfa79-059a-4060-98be-bf5141bd3a69 admin admin] PATH_INFO: `/v3/projects/76cc762f121e46c096aaca327258258c/users/f1fd384cbf4943368ee57e868c37d92c/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:53.960793 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afedfa79-059a-4060-98be-bf5141bd3a69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76cc762f121e46c096aaca327258258c, user_id=f1fd384cbf4943368ee57e868c37d92c, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:53.964348 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afedfa79-059a-4060-98be-bf5141bd3a69 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:54.047502 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1022/2073] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:07:53 2022] PUT /identity/v3/projects/76cc762f121e46c096aaca327258258c/users/f1fd384cbf4943368ee57e868c37d92c/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:07:54.054718 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14380d-985d-4524-8742-0f97b808b0bc None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:54.054897 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14380d-985d-4524-8742-0f97b808b0bc None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:54.055174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14380d-985d-4524-8742-0f97b808b0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:54.078763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-ba14380d-985d-4524-8742-0f97b808b0bc None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:54.080311 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ba14380d-985d-4524-8742-0f97b808b0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:54.165287 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1052/2074] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:54.176301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedfa79-059a-4060-98be-bf5141bd3a69 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:54.182023 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-afedfa79-059a-4060-98be-bf5141bd3a69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:54.260476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedfa79-059a-4060-98be-bf5141bd3a69 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:54.264607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b6518-131b-4122-bfee-740851b97d16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:54.265243 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b6518-131b-4122-bfee-740851b97d16 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:54.265607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b6518-131b-4122-bfee-740851b97d16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:54.265768 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b6518-131b-4122-bfee-740851b97d16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:54.287980 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b6518-131b-4122-bfee-740851b97d16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:54.289700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b6518-131b-4122-bfee-740851b97d16 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:54.302471 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1023/2075] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:07:55.030271 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35acdd8d-b1d2-49a9-82a9-c8debcbe5818 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:55.030829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35acdd8d-b1d2-49a9-82a9-c8debcbe5818 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:55.030829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35acdd8d-b1d2-49a9-82a9-c8debcbe5818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:55.053093 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-35acdd8d-b1d2-49a9-82a9-c8debcbe5818 None None] MFA Rules not processed for user `8b2daf470f45422a8c1bd5cf62efdd9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:55.055899 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-35acdd8d-b1d2-49a9-82a9-c8debcbe5818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:55.132889 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1053/2076] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:56.090750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac86bb-7246-4dfb-be90-21bb63bf12c6 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:56.091350 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac86bb-7246-4dfb-be90-21bb63bf12c6 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:56.091686 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac86bb-7246-4dfb-be90-21bb63bf12c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:56.147805 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-d1ac86bb-7246-4dfb-be90-21bb63bf12c6 None None] MFA Rules not processed for user `f1fd384cbf4943368ee57e868c37d92c`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:56.150212 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-d1ac86bb-7246-4dfb-be90-21bb63bf12c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:56.219035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:56.292453 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1024/2077] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:56.374836 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35acdd8d-b1d2-49a9-82a9-c8debcbe5818 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:56.376381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-35acdd8d-b1d2-49a9-82a9-c8debcbe5818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:56.484475 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35acdd8d-b1d2-49a9-82a9-c8debcbe5818 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:56.489455 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e66bfd-f3a5-4dc1-825d-bd0467e5514f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:56.490712 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e66bfd-f3a5-4dc1-825d-bd0467e5514f service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:56.491304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e66bfd-f3a5-4dc1-825d-bd0467e5514f service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:56.491807 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e66bfd-f3a5-4dc1-825d-bd0467e5514f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:56.502955 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e66bfd-f3a5-4dc1-825d-bd0467e5514f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:56.504436 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e66bfd-f3a5-4dc1-825d-bd0467e5514f service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:56.515260 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1054/2078] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:07:56.609779 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac86bb-7246-4dfb-be90-21bb63bf12c6 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:56.614598 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-d1ac86bb-7246-4dfb-be90-21bb63bf12c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:56.631675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac86bb-7246-4dfb-be90-21bb63bf12c6 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:56.640742 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2342de9-d28f-47e6-a808-6fa9caf5273f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:56.642204 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2342de9-d28f-47e6-a808-6fa9caf5273f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:56.642866 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2342de9-d28f-47e6-a808-6fa9caf5273f admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:56.643363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2342de9-d28f-47e6-a808-6fa9caf5273f admin admin] PATH_INFO: `/v3/users/6d9f61a53ca64d59a1b24420320472b2` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:56.646773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2342de9-d28f-47e6-a808-6fa9caf5273f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d9f61a53ca64d59a1b24420320472b2)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:56.652850 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2342de9-d28f-47e6-a808-6fa9caf5273f admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:56.864105 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b2342de9-d28f-47e6-a808-6fa9caf5273f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d9f61a53ca64d59a1b24420320472b2'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:56.872996 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b2342de9-d28f-47e6-a808-6fa9caf5273f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d9f61a53ca64d59a1b24420320472b2'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:56.873788 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b2342de9-d28f-47e6-a808-6fa9caf5273f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d9f61a53ca64d59a1b24420320472b2'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:56.986609 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b2342de9-d28f-47e6-a808-6fa9caf5273f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d9f61a53ca64d59a1b24420320472b2'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:57.008461 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b2342de9-d28f-47e6-a808-6fa9caf5273f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d9f61a53ca64d59a1b24420320472b2'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:57.019515 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1025/2079] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:07:56 2022] DELETE /identity/v3/users/6d9f61a53ca64d59a1b24420320472b2 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:07:57.026135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c058575-c5c4-405e-bf46-333d868c310d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:57.026424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c058575-c5c4-405e-bf46-333d868c310d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:57.026717 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c058575-c5c4-405e-bf46-333d868c310d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:57.054542 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-3c058575-c5c4-405e-bf46-333d868c310d None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:57.054664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3c058575-c5c4-405e-bf46-333d868c310d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:57.164038 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:57.194647 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1055/2080] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:07:57.203113 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2342de9-d28f-47e6-a808-6fa9caf5273f admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:57.208968 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-b2342de9-d28f-47e6-a808-6fa9caf5273f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:57.348072 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2342de9-d28f-47e6-a808-6fa9caf5273f admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:57.356632 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd9655-4cfc-4253-9c84-c664e9bc6f6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:57.357354 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd9655-4cfc-4253-9c84-c664e9bc6f6a service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:57.357794 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd9655-4cfc-4253-9c84-c664e9bc6f6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:57.358098 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd9655-4cfc-4253-9c84-c664e9bc6f6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:57.371723 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd9655-4cfc-4253-9c84-c664e9bc6f6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:57.373550 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fd9655-4cfc-4253-9c84-c664e9bc6f6a service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:57.390000 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1026/2081] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:07:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:07:57.926317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c058575-c5c4-405e-bf46-333d868c310d None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:57.928645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3c058575-c5c4-405e-bf46-333d868c310d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:57.959056 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c058575-c5c4-405e-bf46-333d868c310d None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:57.966523 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876324f-935d-467e-a42f-fb656de8beee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:57.967530 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876324f-935d-467e-a42f-fb656de8beee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:57.968351 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876324f-935d-467e-a42f-fb656de8beee admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:57.968866 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876324f-935d-467e-a42f-fb656de8beee admin admin] PATH_INFO: `/v3/projects/26323ee7d3014b80986cea144ca68b1c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:57.971009 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5876324f-935d-467e-a42f-fb656de8beee admin admin] RBAC: Authorizing `identity:delete_project(project_id=26323ee7d3014b80986cea144ca68b1c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:57.975164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5876324f-935d-467e-a42f-fb656de8beee admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:58.104644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5876324f-935d-467e-a42f-fb656de8beee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26323ee7d3014b80986cea144ca68b1c', 'request_id': 'req-5876324f-935d-467e-a42f-fb656de8beee'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:58.113498 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5876324f-935d-467e-a42f-fb656de8beee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26323ee7d3014b80986cea144ca68b1c', 'request_id': 'req-5876324f-935d-467e-a42f-fb656de8beee'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:58.198775 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5876324f-935d-467e-a42f-fb656de8beee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26323ee7d3014b80986cea144ca68b1c', 'request_id': 'req-5876324f-935d-467e-a42f-fb656de8beee'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:58.222350 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5876324f-935d-467e-a42f-fb656de8beee admin admin] The token cache is being invalidate because project 26323ee7d3014b80986cea144ca68b1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:07:58.223036 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5876324f-935d-467e-a42f-fb656de8beee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:58.225299 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5876324f-935d-467e-a42f-fb656de8beee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:58.233103 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1056/2082] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:07:57 2022] DELETE /identity/v3/projects/26323ee7d3014b80986cea144ca68b1c => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:58.235201 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd9655-4cfc-4253-9c84-c664e9bc6f6a service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:58.253947 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd9655-4cfc-4253-9c84-c664e9bc6f6a service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:58.316487 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:58.336444 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839d4ec-fab2-4cc8-a8cb-e959e1650d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:58.337455 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839d4ec-fab2-4cc8-a8cb-e959e1650d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:58.337807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839d4ec-fab2-4cc8-a8cb-e959e1650d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:58.338206 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839d4ec-fab2-4cc8-a8cb-e959e1650d05 admin admin] PATH_INFO: `/v3/users/1686f22c057c4727b8e9097ae9bb4aed` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:58.340016 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839d4ec-fab2-4cc8-a8cb-e959e1650d05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1686f22c057c4727b8e9097ae9bb4aed)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:58.342375 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8839d4ec-fab2-4cc8-a8cb-e959e1650d05 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:58.637735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8839d4ec-fab2-4cc8-a8cb-e959e1650d05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1686f22c057c4727b8e9097ae9bb4aed'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:58.648361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8839d4ec-fab2-4cc8-a8cb-e959e1650d05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1686f22c057c4727b8e9097ae9bb4aed'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:58.649439 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8839d4ec-fab2-4cc8-a8cb-e959e1650d05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1686f22c057c4727b8e9097ae9bb4aed'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:58.750390 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8839d4ec-fab2-4cc8-a8cb-e959e1650d05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1686f22c057c4727b8e9097ae9bb4aed'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:58.765081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8839d4ec-fab2-4cc8-a8cb-e959e1650d05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1686f22c057c4727b8e9097ae9bb4aed'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:58.773192 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1027/2083] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:07:58 2022] DELETE /identity/v3/users/1686f22c057c4727b8e9097ae9bb4aed => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:07:59.448983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876324f-935d-467e-a42f-fb656de8beee admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:07:59.448983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5876324f-935d-467e-a42f-fb656de8beee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:59.470261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876324f-935d-467e-a42f-fb656de8beee admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:07:59.670906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65241e1-0147-498c-8f7d-b44876399312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:07:59.672460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65241e1-0147-498c-8f7d-b44876399312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:59.673642 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65241e1-0147-498c-8f7d-b44876399312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:59.674292 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65241e1-0147-498c-8f7d-b44876399312 admin admin] PATH_INFO: `/v3/projects/6343e71c05454ea49b14d2d84d12c62b` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:59.676243 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65241e1-0147-498c-8f7d-b44876399312 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6343e71c05454ea49b14d2d84d12c62b)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:07:59.682046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65241e1-0147-498c-8f7d-b44876399312 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:07:59.692017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742a778-beaf-4536-b171-d806266d959c None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:59.692809 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742a778-beaf-4536-b171-d806266d959c None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:59.692809 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742a778-beaf-4536-b171-d806266d959c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:59.729104 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-4742a778-beaf-4536-b171-d806266d959c None None] MFA Rules not processed for user `f1fd384cbf4943368ee57e868c37d92c`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:59.730411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4742a778-beaf-4536-b171-d806266d959c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:59.866429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1028/2084] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:07:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:07:59.869015 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2c60d-4984-476e-b7ac-09e7c3ff418a None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:07:59.870332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e65241e1-0147-498c-8f7d-b44876399312 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6343e71c05454ea49b14d2d84d12c62b', 'request_id': 'req-e65241e1-0147-498c-8f7d-b44876399312'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:59.870709 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2c60d-4984-476e-b7ac-09e7c3ff418a None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:07:59.871922 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2c60d-4984-476e-b7ac-09e7c3ff418a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:07:59.885735 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e65241e1-0147-498c-8f7d-b44876399312 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6343e71c05454ea49b14d2d84d12c62b', 'request_id': 'req-e65241e1-0147-498c-8f7d-b44876399312'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:59.918908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-7fa2c60d-4984-476e-b7ac-09e7c3ff418a None None] MFA Rules not processed for user `ed8114b7a28741268fdff71c866dbc77`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:07:59.918908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-7fa2c60d-4984-476e-b7ac-09e7c3ff418a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:07:59.967736 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e65241e1-0147-498c-8f7d-b44876399312 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6343e71c05454ea49b14d2d84d12c62b', 'request_id': 'req-e65241e1-0147-498c-8f7d-b44876399312'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:59.978997 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:07:59.988772 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e65241e1-0147-498c-8f7d-b44876399312 admin admin] The token cache is being invalidate because project 6343e71c05454ea49b14d2d84d12c62b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:07:59.992384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e65241e1-0147-498c-8f7d-b44876399312 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:07:59.994680 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e65241e1-0147-498c-8f7d-b44876399312 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:00.000652 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1057/2085] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:07:59 2022] DELETE /identity/v3/projects/6343e71c05454ea49b14d2d84d12c62b => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:08:00.006565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722fcc9-d397-4dbc-b3de-29c8b1fdf99f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:00.007547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722fcc9-d397-4dbc-b3de-29c8b1fdf99f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:00.008144 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1722fcc9-d397-4dbc-b3de-29c8b1fdf99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:00.034286 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-1722fcc9-d397-4dbc-b3de-29c8b1fdf99f None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:08:00.035814 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1722fcc9-d397-4dbc-b3de-29c8b1fdf99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:00.137587 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1058/2086] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:08:00.147493 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bcbbf-2b3c-446d-b807-afdb0a53ae7c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:00.148055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bcbbf-2b3c-446d-b807-afdb0a53ae7c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:00.149939 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1029/2087] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 2 09:07:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:08:00.150535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090bcbbf-2b3c-446d-b807-afdb0a53ae7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:00.165258 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2c60d-4984-476e-b7ac-09e7c3ff418a None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:00.167239 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-7fa2c60d-4984-476e-b7ac-09e7c3ff418a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:00.201588 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:08:00.201855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-090bcbbf-2b3c-446d-b807-afdb0a53ae7c None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:08:00.203357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-090bcbbf-2b3c-446d-b807-afdb0a53ae7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:00.227461 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1059/2088] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:08:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:08:00.228651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bcbbf-2b3c-446d-b807-afdb0a53ae7c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:00.245517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090bcbbf-2b3c-446d-b807-afdb0a53ae7c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:00.253720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101c79b-e684-4d6e-aa17-5a9f7d582e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:00.254569 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101c79b-e684-4d6e-aa17-5a9f7d582e18 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:00.254569 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101c79b-e684-4d6e-aa17-5a9f7d582e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:00.254870 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101c79b-e684-4d6e-aa17-5a9f7d582e18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:00.255537 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101c79b-e684-4d6e-aa17-5a9f7d582e18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:08:00.256013 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101c79b-e684-4d6e-aa17-5a9f7d582e18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:00.257690 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101c79b-e684-4d6e-aa17-5a9f7d582e18 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:00.278900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1060/2089] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:08:00.284380 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101c79b-e684-4d6e-aa17-5a9f7d582e18 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:00.300880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101c79b-e684-4d6e-aa17-5a9f7d582e18 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:00.314190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a7309-7412-43d4-a3f1-a448c29adde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:00.315048 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a7309-7412-43d4-a3f1-a448c29adde9 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:00.315340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a7309-7412-43d4-a3f1-a448c29adde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:00.315656 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a7309-7412-43d4-a3f1-a448c29adde9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:00.316316 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a7309-7412-43d4-a3f1-a448c29adde9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:00.318341 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a7309-7412-43d4-a3f1-a448c29adde9 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:00.331355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2c60d-4984-476e-b7ac-09e7c3ff418a None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:00.343765 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95483c74-4d01-488f-85f4-a1dfcb23d002 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff92ef3b9da74ccab3fbc43f852e3ad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:00.344717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95483c74-4d01-488f-85f4-a1dfcb23d002 service swift] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:00.345187 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95483c74-4d01-488f-85f4-a1dfcb23d002 service swift] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:00.345657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95483c74-4d01-488f-85f4-a1dfcb23d002 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:00.360224 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95483c74-4d01-488f-85f4-a1dfcb23d002 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:00.362850 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95483c74-4d01-488f-85f4-a1dfcb23d002 service swift] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:00.377808 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1030/2090] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 2 09:08:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 02 09:08:00.433132 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1061/2091] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:08:00 2022] POST /identity/v3/projects => generated 377 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:08:00.437235 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95483c74-4d01-488f-85f4-a1dfcb23d002 service swift] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:00.451453 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95483c74-4d01-488f-85f4-a1dfcb23d002 service swift] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:00.538527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157dd5b-32ad-45aa-b0d6-e01caa4e3ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:00.538527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157dd5b-32ad-45aa-b0d6-e01caa4e3ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:00.538527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157dd5b-32ad-45aa-b0d6-e01caa4e3ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:00.538527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157dd5b-32ad-45aa-b0d6-e01caa4e3ca4 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:00.539821 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3157dd5b-32ad-45aa-b0d6-e01caa4e3ca4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:00.539821 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3157dd5b-32ad-45aa-b0d6-e01caa4e3ca4 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:00.704717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1031/2092] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:08:00 2022] POST /identity/v3/users => generated 289 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:08:00.707101 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a7309-7412-43d4-a3f1-a448c29adde9 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:00.723491 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a7309-7412-43d4-a3f1-a448c29adde9 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:00.730067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea58765-411f-41c3-9ad6-d9b8ab096498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:00.730965 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea58765-411f-41c3-9ad6-d9b8ab096498 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:00.731396 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea58765-411f-41c3-9ad6-d9b8ab096498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:00.731827 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea58765-411f-41c3-9ad6-d9b8ab096498 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:00.732711 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea58765-411f-41c3-9ad6-d9b8ab096498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:00.737056 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea58765-411f-41c3-9ad6-d9b8ab096498 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:00.805752 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157dd5b-32ad-45aa-b0d6-e01caa4e3ca4 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:00.807083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1062/2093] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:08:00.856308 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157dd5b-32ad-45aa-b0d6-e01caa4e3ca4 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:00.872644 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3f946f-88d8-458c-876a-71e63c02a83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:00.873909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f946f-88d8-458c-876a-71e63c02a83e admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:00.874366 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f946f-88d8-458c-876a-71e63c02a83e admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:00.874885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3f946f-88d8-458c-876a-71e63c02a83e admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:00.876485 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f946f-88d8-458c-876a-71e63c02a83e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:00.883735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3f946f-88d8-458c-876a-71e63c02a83e admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:00.910792 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea58765-411f-41c3-9ad6-d9b8ab096498 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:00.912192 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1032/2094] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:08:00.941353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea58765-411f-41c3-9ad6-d9b8ab096498 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:00.965957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184461b-9108-4282-a04a-a2c3aba48cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:00.967334 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184461b-9108-4282-a04a-a2c3aba48cd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:00.967857 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184461b-9108-4282-a04a-a2c3aba48cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:00.968338 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5184461b-9108-4282-a04a-a2c3aba48cd1 admin admin] PATH_INFO: `/v3/projects/9fad90825ade447992b855635af31169/users/45d6d1b60eda44139a2dae7de22c97b4/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:01.020253 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5184461b-9108-4282-a04a-a2c3aba48cd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fad90825ade447992b855635af31169, user_id=45d6d1b60eda44139a2dae7de22c97b4, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:01.027443 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5184461b-9108-4282-a04a-a2c3aba48cd1 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:01.172485 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1063/2095] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:08:00 2022] PUT /identity/v3/projects/9fad90825ade447992b855635af31169/users/45d6d1b60eda44139a2dae7de22c97b4/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:08:01.191969 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af93131-58c6-4011-b79b-62fe423c29f1 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:01.200963 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af93131-58c6-4011-b79b-62fe423c29f1 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:01.200963 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af93131-58c6-4011-b79b-62fe423c29f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:01.314062 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-4af93131-58c6-4011-b79b-62fe423c29f1 None None] MFA Rules not processed for user `45d6d1b60eda44139a2dae7de22c97b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:08:01.315520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4af93131-58c6-4011-b79b-62fe423c29f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:01.558691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1033/2096] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:08:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:08:01.563720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184461b-9108-4282-a04a-a2c3aba48cd1 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:01.687345 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5184461b-9108-4282-a04a-a2c3aba48cd1 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:01.696360 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b640a-dc9d-449c-8dbb-370c11842455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:01.697437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b640a-dc9d-449c-8dbb-370c11842455 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:01.698708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b640a-dc9d-449c-8dbb-370c11842455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:01.699361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b640a-dc9d-449c-8dbb-370c11842455 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:01.700265 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b640a-dc9d-449c-8dbb-370c11842455 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:01.703304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b640a-dc9d-449c-8dbb-370c11842455 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:01.803363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af93131-58c6-4011-b79b-62fe423c29f1 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:01.803941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1064/2097] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:08:01 2022] POST /identity/v3/projects => generated 377 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:08:01.827275 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af93131-58c6-4011-b79b-62fe423c29f1 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:01.873613 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:08:02.014793 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde3d09-1765-43d4-b9bc-c3b0518da727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:02.017202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde3d09-1765-43d4-b9bc-c3b0518da727 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:02.017500 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde3d09-1765-43d4-b9bc-c3b0518da727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:02.017783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dde3d09-1765-43d4-b9bc-c3b0518da727 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:02.018586 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde3d09-1765-43d4-b9bc-c3b0518da727 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:02.039236 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dde3d09-1765-43d4-b9bc-c3b0518da727 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:02.173169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1034/2098] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:08:01 2022] POST /identity/v3/users => generated 295 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:08:02.177982 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b640a-dc9d-449c-8dbb-370c11842455 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:02.204561 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b640a-dc9d-449c-8dbb-370c11842455 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:02.215635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bc7cd-0f61-4bcb-b539-6a3d95bc71da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:02.217016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bc7cd-0f61-4bcb-b539-6a3d95bc71da admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:02.217520 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bc7cd-0f61-4bcb-b539-6a3d95bc71da admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:02.218039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bc7cd-0f61-4bcb-b539-6a3d95bc71da admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:02.219308 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7bc7cd-0f61-4bcb-b539-6a3d95bc71da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:02.221928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7bc7cd-0f61-4bcb-b539-6a3d95bc71da admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:02.227659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:08:02.248293 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1065/2099] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:08:02.250422 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde3d09-1765-43d4-b9bc-c3b0518da727 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:02.267180 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dde3d09-1765-43d4-b9bc-c3b0518da727 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:02.281314 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a7a4dc-64da-41ba-a1d0-646cab3720e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:02.283057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7a4dc-64da-41ba-a1d0-646cab3720e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:02.288077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7a4dc-64da-41ba-a1d0-646cab3720e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:02.288924 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7a4dc-64da-41ba-a1d0-646cab3720e8 admin admin] PATH_INFO: `/v3/projects/febf5943abdd4a1ba0ee51f80c395e68/users/7448de0fd25842f6909e8888f2a8b98c/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:02.349067 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a7a4dc-64da-41ba-a1d0-646cab3720e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=febf5943abdd4a1ba0ee51f80c395e68, user_id=7448de0fd25842f6909e8888f2a8b98c, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:02.363479 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a7a4dc-64da-41ba-a1d0-646cab3720e8 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:02.497585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1035/2100] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:08:02 2022] PUT /identity/v3/projects/febf5943abdd4a1ba0ee51f80c395e68/users/7448de0fd25842f6909e8888f2a8b98c/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:08:02.502274 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bc7cd-0f61-4bcb-b539-6a3d95bc71da admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:02.514951 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bc7cd-0f61-4bcb-b539-6a3d95bc71da admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:02.631074 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4edadaa-fc49-44b5-9fcb-02912ed44cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:02.631852 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4edadaa-fc49-44b5-9fcb-02912ed44cda admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:02.632646 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4edadaa-fc49-44b5-9fcb-02912ed44cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:02.632646 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4edadaa-fc49-44b5-9fcb-02912ed44cda admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:02.634068 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4edadaa-fc49-44b5-9fcb-02912ed44cda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:02.636178 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4edadaa-fc49-44b5-9fcb-02912ed44cda admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:02.648259 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1066/2101] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:08:02.653039 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a7a4dc-64da-41ba-a1d0-646cab3720e8 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:02.664311 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a7a4dc-64da-41ba-a1d0-646cab3720e8 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:02.670929 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e1760-c11b-444e-aa67-3c6ba54980de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:02.672173 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e1760-c11b-444e-aa67-3c6ba54980de admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:02.672432 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e1760-c11b-444e-aa67-3c6ba54980de admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:02.672759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e1760-c11b-444e-aa67-3c6ba54980de admin admin] PATH_INFO: `/v3/projects/febf5943abdd4a1ba0ee51f80c395e68/users/7448de0fd25842f6909e8888f2a8b98c/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:02.675874 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e1760-c11b-444e-aa67-3c6ba54980de admin admin] RBAC: Authorizing `identity:create_grant(project_id=febf5943abdd4a1ba0ee51f80c395e68, user_id=7448de0fd25842f6909e8888f2a8b98c, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:02.679149 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e1760-c11b-444e-aa67-3c6ba54980de admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:02.706976 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec896af4-69f4-4ca5-80d1-fbefbfeb086d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:02.707315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec896af4-69f4-4ca5-80d1-fbefbfeb086d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:02.707573 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec896af4-69f4-4ca5-80d1-fbefbfeb086d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:02.711186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1036/2102] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:08:02 2022] PUT /identity/v3/projects/febf5943abdd4a1ba0ee51f80c395e68/users/7448de0fd25842f6909e8888f2a8b98c/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:08:02.749007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-ec896af4-69f4-4ca5-80d1-fbefbfeb086d None None] MFA Rules not processed for user `7448de0fd25842f6909e8888f2a8b98c`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:08:02.750895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ec896af4-69f4-4ca5-80d1-fbefbfeb086d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:02.882745 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1067/2103] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:08:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:08:02.896470 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e1760-c11b-444e-aa67-3c6ba54980de admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:02.900682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-d03e1760-c11b-444e-aa67-3c6ba54980de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:03.125711 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e1760-c11b-444e-aa67-3c6ba54980de admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:03.140461 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3eea4-61bb-4597-8857-8159618cbd45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:03.145048 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3eea4-61bb-4597-8857-8159618cbd45 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:03.146763 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3eea4-61bb-4597-8857-8159618cbd45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:03.147329 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3eea4-61bb-4597-8857-8159618cbd45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:03.399875 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c3eea4-61bb-4597-8857-8159618cbd45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:03.402003 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c3eea4-61bb-4597-8857-8159618cbd45 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:03.426137 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1037/2104] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:08:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:08:04.612047 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d9821-d8cc-4c42-8b03-65860cb7c5f8 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:04.612047 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d9821-d8cc-4c42-8b03-65860cb7c5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:04.612047 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d9821-d8cc-4c42-8b03-65860cb7c5f8 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:04.613282 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1068/2105] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 2 09:08:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:08:04.644777 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76de80-3c7d-41f5-b788-9ae813c04418 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:04.645598 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76de80-3c7d-41f5-b788-9ae813c04418 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:04.646330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76de80-3c7d-41f5-b788-9ae813c04418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:04.679849 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-cc76de80-3c7d-41f5-b788-9ae813c04418 None None] MFA Rules not processed for user `cad5cc9927374836b0cc74db85b686fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:08:04.681785 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-cc76de80-3c7d-41f5-b788-9ae813c04418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:04.714645 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:08:04.790577 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1038/2106] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 2 09:08:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:08:04.802136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d9821-d8cc-4c42-8b03-65860cb7c5f8 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:04.804155 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e88d9821-d8cc-4c42-8b03-65860cb7c5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:04.827174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d9821-d8cc-4c42-8b03-65860cb7c5f8 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:04.838099 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac564b-258c-4565-80b5-0557d8311fb8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad5cc9927374836b0cc74db85b686fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:04.838099 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac564b-258c-4565-80b5-0557d8311fb8 service placement] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:04.838099 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac564b-258c-4565-80b5-0557d8311fb8 service placement] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:04.838099 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac564b-258c-4565-80b5-0557d8311fb8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:04.839677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76de80-3c7d-41f5-b788-9ae813c04418 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:04.841392 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-cc76de80-3c7d-41f5-b788-9ae813c04418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:04.844968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dac564b-258c-4565-80b5-0557d8311fb8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:04.846631 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dac564b-258c-4565-80b5-0557d8311fb8 service placement] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:04.861336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1069/2107] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:08:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:08:04.949152 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76de80-3c7d-41f5-b788-9ae813c04418 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:04.954093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f97ff-288a-41e4-94a2-8436bdd9cd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:04.954758 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f97ff-288a-41e4-94a2-8436bdd9cd36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:04.955289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f97ff-288a-41e4-94a2-8436bdd9cd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:04.955567 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f97ff-288a-41e4-94a2-8436bdd9cd36 admin admin] PATH_INFO: `/v3/users/3b9d9cfdcaac473786d3017f73163274` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:04.956794 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f97ff-288a-41e4-94a2-8436bdd9cd36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b9d9cfdcaac473786d3017f73163274)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:04.958783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f97ff-288a-41e4-94a2-8436bdd9cd36 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:05.152424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-028f97ff-288a-41e4-94a2-8436bdd9cd36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b9d9cfdcaac473786d3017f73163274'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:05.161544 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-028f97ff-288a-41e4-94a2-8436bdd9cd36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b9d9cfdcaac473786d3017f73163274'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:05.162822 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-028f97ff-288a-41e4-94a2-8436bdd9cd36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b9d9cfdcaac473786d3017f73163274'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:05.243913 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-028f97ff-288a-41e4-94a2-8436bdd9cd36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b9d9cfdcaac473786d3017f73163274'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:05.256796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-028f97ff-288a-41e4-94a2-8436bdd9cd36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b9d9cfdcaac473786d3017f73163274'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:05.263442 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1039/2108] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:08:04 2022] DELETE /identity/v3/users/3b9d9cfdcaac473786d3017f73163274 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:08:05.568284 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac564b-258c-4565-80b5-0557d8311fb8 service placement] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:05.594633 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac564b-258c-4565-80b5-0557d8311fb8 service placement] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:05.694541 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d9ad9-d860-4af5-aa4f-5560ab44c168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:05.695281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d9ad9-d860-4af5-aa4f-5560ab44c168 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:05.695615 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d9ad9-d860-4af5-aa4f-5560ab44c168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:05.696450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d9ad9-d860-4af5-aa4f-5560ab44c168 admin admin] PATH_INFO: `/v3/projects/b198992cc0014c6a9c25374d24fb01a9` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:05.699741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d9ad9-d860-4af5-aa4f-5560ab44c168 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b198992cc0014c6a9c25374d24fb01a9)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:05.701743 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d9ad9-d860-4af5-aa4f-5560ab44c168 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:05.714027 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:08:05.844531 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-983d9ad9-d860-4af5-aa4f-5560ab44c168 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b198992cc0014c6a9c25374d24fb01a9', 'request_id': 'req-983d9ad9-d860-4af5-aa4f-5560ab44c168'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:05.875638 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-983d9ad9-d860-4af5-aa4f-5560ab44c168 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b198992cc0014c6a9c25374d24fb01a9', 'request_id': 'req-983d9ad9-d860-4af5-aa4f-5560ab44c168'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:05.946639 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-983d9ad9-d860-4af5-aa4f-5560ab44c168 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b198992cc0014c6a9c25374d24fb01a9', 'request_id': 'req-983d9ad9-d860-4af5-aa4f-5560ab44c168'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:05.971864 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-983d9ad9-d860-4af5-aa4f-5560ab44c168 admin admin] The token cache is being invalidate because project b198992cc0014c6a9c25374d24fb01a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:08:05.972159 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-983d9ad9-d860-4af5-aa4f-5560ab44c168 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:05.973220 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-983d9ad9-d860-4af5-aa4f-5560ab44c168 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:05.975367 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1070/2109] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:08:05 2022] DELETE /identity/v3/projects/b198992cc0014c6a9c25374d24fb01a9 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:08:05.980814 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b969d-3cb1-4f31-8c13-63f774d4bc1c None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:05.982313 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b969d-3cb1-4f31-8c13-63f774d4bc1c None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:05.982313 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b969d-3cb1-4f31-8c13-63f774d4bc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:06.008904 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-106b969d-3cb1-4f31-8c13-63f774d4bc1c None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:08:06.010729 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-106b969d-3cb1-4f31-8c13-63f774d4bc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:06.066424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:08:06.130312 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1040/2110] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:08:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:08:06.136273 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c4fec-8fcd-4c43-afa2-b48bc8707299 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:06.136618 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c4fec-8fcd-4c43-afa2-b48bc8707299 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:06.136901 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c4fec-8fcd-4c43-afa2-b48bc8707299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:06.174298 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-227c4fec-8fcd-4c43-afa2-b48bc8707299 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:08:06.175851 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-227c4fec-8fcd-4c43-afa2-b48bc8707299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:06.189138 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1071/2111] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:08:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:08:06.193129 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b969d-3cb1-4f31-8c13-63f774d4bc1c None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:06.209512 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b969d-3cb1-4f31-8c13-63f774d4bc1c None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:06.218654 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c41ee-06fa-4eda-b855-f652bfbfbe38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:06.220223 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c41ee-06fa-4eda-b855-f652bfbfbe38 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:06.220507 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c41ee-06fa-4eda-b855-f652bfbfbe38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:06.220832 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c41ee-06fa-4eda-b855-f652bfbfbe38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:06.222198 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c41ee-06fa-4eda-b855-f652bfbfbe38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:08:06.222758 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c41ee-06fa-4eda-b855-f652bfbfbe38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:06.226280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c41ee-06fa-4eda-b855-f652bfbfbe38 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:06.254978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1041/2112] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:08:06.260530 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c4fec-8fcd-4c43-afa2-b48bc8707299 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:06.272039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c4fec-8fcd-4c43-afa2-b48bc8707299 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:06.286937 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27356f3-ed0b-4487-b386-7e0125d2ef28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:06.287958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27356f3-ed0b-4487-b386-7e0125d2ef28 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:06.288427 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27356f3-ed0b-4487-b386-7e0125d2ef28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:06.288892 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27356f3-ed0b-4487-b386-7e0125d2ef28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:06.293443 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27356f3-ed0b-4487-b386-7e0125d2ef28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:06.296157 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27356f3-ed0b-4487-b386-7e0125d2ef28 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:06.442836 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1072/2113] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:08:06 2022] POST /identity/v3/projects => generated 365 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:08:06.448705 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c41ee-06fa-4eda-b855-f652bfbfbe38 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:06.464717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c41ee-06fa-4eda-b855-f652bfbfbe38 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:06.565717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631dfde-6cb2-4414-9e57-42ab322b52d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:06.606690 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631dfde-6cb2-4414-9e57-42ab322b52d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:06.606690 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631dfde-6cb2-4414-9e57-42ab322b52d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:06.606690 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631dfde-6cb2-4414-9e57-42ab322b52d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:06.606690 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c631dfde-6cb2-4414-9e57-42ab322b52d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:06.606690 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c631dfde-6cb2-4414-9e57-42ab322b52d9 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:07.657971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1042/2114] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:08:06 2022] POST /identity/v3/users => generated 283 bytes in 1210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:08:07.665680 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27356f3-ed0b-4487-b386-7e0125d2ef28 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:07.683970 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27356f3-ed0b-4487-b386-7e0125d2ef28 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:07.689131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e53243f-a7c5-4b21-93bd-83a07e16e929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:07.689769 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53243f-a7c5-4b21-93bd-83a07e16e929 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:07.690086 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53243f-a7c5-4b21-93bd-83a07e16e929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:07.690364 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53243f-a7c5-4b21-93bd-83a07e16e929 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:07.691075 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e53243f-a7c5-4b21-93bd-83a07e16e929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:07.692736 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e53243f-a7c5-4b21-93bd-83a07e16e929 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:07.705976 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1073/2115] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:08:07.709958 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631dfde-6cb2-4414-9e57-42ab322b52d9 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:07.725162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631dfde-6cb2-4414-9e57-42ab322b52d9 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:07.737954 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c9dd20-d73e-4379-9188-bdd02f4b1a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:07.738842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9dd20-d73e-4379-9188-bdd02f4b1a79 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:07.740025 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9dd20-d73e-4379-9188-bdd02f4b1a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:07.740430 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9dd20-d73e-4379-9188-bdd02f4b1a79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:07.741505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c9dd20-d73e-4379-9188-bdd02f4b1a79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:07.743772 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c9dd20-d73e-4379-9188-bdd02f4b1a79 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:07.760876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1043/2116] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:08:07 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:08:07.761651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e53243f-a7c5-4b21-93bd-83a07e16e929 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:07.785243 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e53243f-a7c5-4b21-93bd-83a07e16e929 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:07.790317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e4dcb-5d48-4379-bf23-e670a5efd489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:07.790999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e4dcb-5d48-4379-bf23-e670a5efd489 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:07.791310 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e4dcb-5d48-4379-bf23-e670a5efd489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:07.791590 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e4dcb-5d48-4379-bf23-e670a5efd489 admin admin] PATH_INFO: `/v3/projects/16feb149df3345fa9f5fc8dc0bb39e6d/users/661187fc383e4d12b16e8b8d79abf64b/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:07.809532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e4dcb-5d48-4379-bf23-e670a5efd489 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16feb149df3345fa9f5fc8dc0bb39e6d, user_id=661187fc383e4d12b16e8b8d79abf64b, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:07.812304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e4dcb-5d48-4379-bf23-e670a5efd489 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:07.927987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1074/2117] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:08:07 2022] PUT /identity/v3/projects/16feb149df3345fa9f5fc8dc0bb39e6d/users/661187fc383e4d12b16e8b8d79abf64b/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:08:07.939102 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a198689-71d1-41a0-bb9f-daef189125bb None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:07.939393 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a198689-71d1-41a0-bb9f-daef189125bb None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:07.939670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a198689-71d1-41a0-bb9f-daef189125bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:07.974805 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-6a198689-71d1-41a0-bb9f-daef189125bb None None] MFA Rules not processed for user `661187fc383e4d12b16e8b8d79abf64b`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:08:07.976027 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-6a198689-71d1-41a0-bb9f-daef189125bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:08.064316 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1044/2118] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:08:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:08:08.068933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e4dcb-5d48-4379-bf23-e670a5efd489 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:08.168541 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e4dcb-5d48-4379-bf23-e670a5efd489 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:08.174005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac9167-c5e2-4879-94b3-2bd2c633ad45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:08.174645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac9167-c5e2-4879-94b3-2bd2c633ad45 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:08.174938 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac9167-c5e2-4879-94b3-2bd2c633ad45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:08.175262 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ac9167-c5e2-4879-94b3-2bd2c633ad45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:08.176001 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac9167-c5e2-4879-94b3-2bd2c633ad45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:08.177968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ac9167-c5e2-4879-94b3-2bd2c633ad45 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:08.338117 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1075/2119] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:08:08 2022] POST /identity/v3/projects => generated 363 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:08:08.342831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a198689-71d1-41a0-bb9f-daef189125bb None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:08.370166 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a198689-71d1-41a0-bb9f-daef189125bb None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:08.464433 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba03af4-8426-4eb3-ab8c-d6e53ef91ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:08.465438 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba03af4-8426-4eb3-ab8c-d6e53ef91ccc admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:08.465733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba03af4-8426-4eb3-ab8c-d6e53ef91ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:08.466114 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba03af4-8426-4eb3-ab8c-d6e53ef91ccc admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:08.466819 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba03af4-8426-4eb3-ab8c-d6e53ef91ccc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:08.468776 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba03af4-8426-4eb3-ab8c-d6e53ef91ccc admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:08.474659 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:08:08.600044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1045/2120] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:08:08 2022] POST /identity/v3/users => generated 288 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:08:08.604716 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac9167-c5e2-4879-94b3-2bd2c633ad45 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:08.617759 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ac9167-c5e2-4879-94b3-2bd2c633ad45 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:08.622732 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9619c-2b47-4e73-8cac-8183c1b07b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:08.623363 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9619c-2b47-4e73-8cac-8183c1b07b22 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:08.623672 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9619c-2b47-4e73-8cac-8183c1b07b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:08.623950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9619c-2b47-4e73-8cac-8183c1b07b22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:08.624649 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9619c-2b47-4e73-8cac-8183c1b07b22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:08.626327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa9619c-2b47-4e73-8cac-8183c1b07b22 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:08.637673 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1076/2121] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:08:08.642809 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba03af4-8426-4eb3-ab8c-d6e53ef91ccc admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:08.664981 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba03af4-8426-4eb3-ab8c-d6e53ef91ccc admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:08.679534 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc7352-ab24-4b7a-99a0-47a2dcb828ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:08.680765 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc7352-ab24-4b7a-99a0-47a2dcb828ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:08.681043 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc7352-ab24-4b7a-99a0-47a2dcb828ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:08.681313 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc7352-ab24-4b7a-99a0-47a2dcb828ff admin admin] PATH_INFO: `/v3/projects/88e506f9b86546ef8ba6dff6bccc9d00/users/f68f2410b2b4477582ee916522662f47/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:08.704096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc7352-ab24-4b7a-99a0-47a2dcb828ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=88e506f9b86546ef8ba6dff6bccc9d00, user_id=f68f2410b2b4477582ee916522662f47, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:08.707377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc7352-ab24-4b7a-99a0-47a2dcb828ff admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:08.808840 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1046/2122] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:08:08 2022] PUT /identity/v3/projects/88e506f9b86546ef8ba6dff6bccc9d00/users/f68f2410b2b4477582ee916522662f47/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:08:08.810101 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9619c-2b47-4e73-8cac-8183c1b07b22 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:08.822619 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9619c-2b47-4e73-8cac-8183c1b07b22 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:08.828850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:08:08.891345 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfc3f10-fee8-44fd-bb3e-5a6286787270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:08.892083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc3f10-fee8-44fd-bb3e-5a6286787270 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:08.892436 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc3f10-fee8-44fd-bb3e-5a6286787270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:08.892804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfc3f10-fee8-44fd-bb3e-5a6286787270 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:08.893688 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfc3f10-fee8-44fd-bb3e-5a6286787270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:08.895380 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfc3f10-fee8-44fd-bb3e-5a6286787270 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:08.908154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1077/2123] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:08:08.912755 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc7352-ab24-4b7a-99a0-47a2dcb828ff admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:08.945162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc7352-ab24-4b7a-99a0-47a2dcb828ff admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:08.950374 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22b4bc-fc35-414b-9ca6-94962a55b1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:08.951057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22b4bc-fc35-414b-9ca6-94962a55b1cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:08.951338 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22b4bc-fc35-414b-9ca6-94962a55b1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:08.951619 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de22b4bc-fc35-414b-9ca6-94962a55b1cd admin admin] PATH_INFO: `/v3/projects/88e506f9b86546ef8ba6dff6bccc9d00/users/f68f2410b2b4477582ee916522662f47/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:08.953627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de22b4bc-fc35-414b-9ca6-94962a55b1cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=88e506f9b86546ef8ba6dff6bccc9d00, user_id=f68f2410b2b4477582ee916522662f47, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:08.956956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de22b4bc-fc35-414b-9ca6-94962a55b1cd admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:08.975504 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1047/2124] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:08:08 2022] PUT /identity/v3/projects/88e506f9b86546ef8ba6dff6bccc9d00/users/f68f2410b2b4477582ee916522662f47/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:08:08.988720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bd5b8-4977-4f7d-a152-ba30ac6b1901 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:08.989352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bd5b8-4977-4f7d-a152-ba30ac6b1901 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:08.989600 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bd5b8-4977-4f7d-a152-ba30ac6b1901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:09.028012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-0a1bd5b8-4977-4f7d-a152-ba30ac6b1901 None None] MFA Rules not processed for user `f68f2410b2b4477582ee916522662f47`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:08:09.029260 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0a1bd5b8-4977-4f7d-a152-ba30ac6b1901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:09.122716 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1078/2125] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:08:08 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:08:09.147572 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22b4bc-fc35-414b-9ca6-94962a55b1cd admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:09.149977 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-de22b4bc-fc35-414b-9ca6-94962a55b1cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:09.243017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de22b4bc-fc35-414b-9ca6-94962a55b1cd admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:09.247247 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a7bb1-274a-455d-82d5-92eb684d2ce1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:09.248183 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a7bb1-274a-455d-82d5-92eb684d2ce1 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:09.248469 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a7bb1-274a-455d-82d5-92eb684d2ce1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:09.248750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a7bb1-274a-455d-82d5-92eb684d2ce1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:09.261729 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a7bb1-274a-455d-82d5-92eb684d2ce1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:09.263230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a7bb1-274a-455d-82d5-92eb684d2ce1 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:09.274852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1048/2126] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:08:09 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:08:10.546906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bd5b8-4977-4f7d-a152-ba30ac6b1901 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:10.611064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bd5b8-4977-4f7d-a152-ba30ac6b1901 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:10.615796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b3e1b-16b3-4762-bbb4-3d86b646b546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:10.616443 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b3e1b-16b3-4762-bbb4-3d86b646b546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:10.616729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b3e1b-16b3-4762-bbb4-3d86b646b546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:10.617008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b3e1b-16b3-4762-bbb4-3d86b646b546 admin admin] PATH_INFO: `/v3/users/661187fc383e4d12b16e8b8d79abf64b` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:10.618882 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b3e1b-16b3-4762-bbb4-3d86b646b546 admin admin] RBAC: Authorizing `identity:delete_user(user_id=661187fc383e4d12b16e8b8d79abf64b)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:10.620810 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b3e1b-16b3-4762-bbb4-3d86b646b546 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:10.821851 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:08:10.837752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-2c1b3e1b-16b3-4762-bbb4-3d86b646b546 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '661187fc383e4d12b16e8b8d79abf64b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:10.844918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-2c1b3e1b-16b3-4762-bbb4-3d86b646b546 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '661187fc383e4d12b16e8b8d79abf64b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:10.845718 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-2c1b3e1b-16b3-4762-bbb4-3d86b646b546 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '661187fc383e4d12b16e8b8d79abf64b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:10.858773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-2c1b3e1b-16b3-4762-bbb4-3d86b646b546 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '661187fc383e4d12b16e8b8d79abf64b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:10.859627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-2c1b3e1b-16b3-4762-bbb4-3d86b646b546 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '661187fc383e4d12b16e8b8d79abf64b'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:10.951518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1079/2127] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:08:10 2022] DELETE /identity/v3/users/661187fc383e4d12b16e8b8d79abf64b => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:08:10.956580 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba9ded-914c-44a5-92c7-c69cbc4c5054 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:10.958164 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba9ded-914c-44a5-92c7-c69cbc4c5054 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:10.958602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba9ded-914c-44a5-92c7-c69cbc4c5054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:10.985776 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-63ba9ded-914c-44a5-92c7-c69cbc4c5054 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:08:10.987236 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-63ba9ded-914c-44a5-92c7-c69cbc4c5054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:11.069540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1049/2128] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:08:11.080997 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b3e1b-16b3-4762-bbb4-3d86b646b546 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:11.082673 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2c1b3e1b-16b3-4762-bbb4-3d86b646b546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:11.182362 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b3e1b-16b3-4762-bbb4-3d86b646b546 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:11.185857 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c4de1-5e98-4764-b77d-27f0d97eae50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:11.186501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c4de1-5e98-4764-b77d-27f0d97eae50 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:11.188331 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c4de1-5e98-4764-b77d-27f0d97eae50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:11.188331 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c4de1-5e98-4764-b77d-27f0d97eae50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:11.197397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c4de1-5e98-4764-b77d-27f0d97eae50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:11.199052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c4de1-5e98-4764-b77d-27f0d97eae50 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:11.209435 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1080/2129] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:08:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:08:11.607724 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba9ded-914c-44a5-92c7-c69cbc4c5054 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:11.609218 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-63ba9ded-914c-44a5-92c7-c69cbc4c5054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:11.631416 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba9ded-914c-44a5-92c7-c69cbc4c5054 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:11.641259 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169a112-4a58-4ea6-91b9-603cd42aac59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:11.642359 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169a112-4a58-4ea6-91b9-603cd42aac59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:11.643454 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169a112-4a58-4ea6-91b9-603cd42aac59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:11.643924 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2169a112-4a58-4ea6-91b9-603cd42aac59 admin admin] PATH_INFO: `/v3/projects/16feb149df3345fa9f5fc8dc0bb39e6d` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:11.645516 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169a112-4a58-4ea6-91b9-603cd42aac59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16feb149df3345fa9f5fc8dc0bb39e6d)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:11.654841 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2169a112-4a58-4ea6-91b9-603cd42aac59 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:11.686922 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:08:11.800927 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-2169a112-4a58-4ea6-91b9-603cd42aac59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16feb149df3345fa9f5fc8dc0bb39e6d', 'request_id': 'req-2169a112-4a58-4ea6-91b9-603cd42aac59'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:11.809575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-2169a112-4a58-4ea6-91b9-603cd42aac59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16feb149df3345fa9f5fc8dc0bb39e6d', 'request_id': 'req-2169a112-4a58-4ea6-91b9-603cd42aac59'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:11.818429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-2169a112-4a58-4ea6-91b9-603cd42aac59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16feb149df3345fa9f5fc8dc0bb39e6d', 'request_id': 'req-2169a112-4a58-4ea6-91b9-603cd42aac59'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:11.933070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-2169a112-4a58-4ea6-91b9-603cd42aac59 admin admin] The token cache is being invalidate because project 16feb149df3345fa9f5fc8dc0bb39e6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:08:11.933495 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-2169a112-4a58-4ea6-91b9-603cd42aac59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:11.934114 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-2169a112-4a58-4ea6-91b9-603cd42aac59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:11.935863 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1050/2130] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:08:11 2022] DELETE /identity/v3/projects/16feb149df3345fa9f5fc8dc0bb39e6d => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:08:11.940715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c4de1-5e98-4764-b77d-27f0d97eae50 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:11.952501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c4de1-5e98-4764-b77d-27f0d97eae50 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:12.047232 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7f7f54-8adb-48b7-afd2-1b7519c1f585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:12.053618 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f7f54-8adb-48b7-afd2-1b7519c1f585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:12.053618 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f7f54-8adb-48b7-afd2-1b7519c1f585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:12.053618 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7f7f54-8adb-48b7-afd2-1b7519c1f585 admin admin] PATH_INFO: `/v3/users/f68f2410b2b4477582ee916522662f47` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:12.053618 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f7f54-8adb-48b7-afd2-1b7519c1f585 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f68f2410b2b4477582ee916522662f47)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:12.053618 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7f7f54-8adb-48b7-afd2-1b7519c1f585 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:12.276556 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fc7f7f54-8adb-48b7-afd2-1b7519c1f585 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f68f2410b2b4477582ee916522662f47'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:12.282206 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:08:12.288222 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fc7f7f54-8adb-48b7-afd2-1b7519c1f585 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f68f2410b2b4477582ee916522662f47'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:12.288222 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fc7f7f54-8adb-48b7-afd2-1b7519c1f585 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f68f2410b2b4477582ee916522662f47'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:12.302772 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fc7f7f54-8adb-48b7-afd2-1b7519c1f585 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f68f2410b2b4477582ee916522662f47'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:12.303511 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fc7f7f54-8adb-48b7-afd2-1b7519c1f585 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f68f2410b2b4477582ee916522662f47'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:12.380756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1081/2131] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:08:11 2022] DELETE /identity/v3/users/f68f2410b2b4477582ee916522662f47 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:08:12.897155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169a112-4a58-4ea6-91b9-603cd42aac59 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:12.899708 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-2169a112-4a58-4ea6-91b9-603cd42aac59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:12.918860 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2169a112-4a58-4ea6-91b9-603cd42aac59 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:13.005663 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9924c-65e8-4d8f-91d0-1988c828bec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:13.006425 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9924c-65e8-4d8f-91d0-1988c828bec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:13.006735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9924c-65e8-4d8f-91d0-1988c828bec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:13.007041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9924c-65e8-4d8f-91d0-1988c828bec2 admin admin] PATH_INFO: `/v3/projects/88e506f9b86546ef8ba6dff6bccc9d00` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:13.009455 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f9924c-65e8-4d8f-91d0-1988c828bec2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88e506f9b86546ef8ba6dff6bccc9d00)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:13.011472 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f9924c-65e8-4d8f-91d0-1988c828bec2 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:13.214479 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-14f9924c-65e8-4d8f-91d0-1988c828bec2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88e506f9b86546ef8ba6dff6bccc9d00', 'request_id': 'req-14f9924c-65e8-4d8f-91d0-1988c828bec2'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:13.222752 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-14f9924c-65e8-4d8f-91d0-1988c828bec2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88e506f9b86546ef8ba6dff6bccc9d00', 'request_id': 'req-14f9924c-65e8-4d8f-91d0-1988c828bec2'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:13.230201 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-14f9924c-65e8-4d8f-91d0-1988c828bec2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88e506f9b86546ef8ba6dff6bccc9d00', 'request_id': 'req-14f9924c-65e8-4d8f-91d0-1988c828bec2'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:13.317451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-14f9924c-65e8-4d8f-91d0-1988c828bec2 admin admin] The token cache is being invalidate because project 88e506f9b86546ef8ba6dff6bccc9d00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:08:13.319630 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-14f9924c-65e8-4d8f-91d0-1988c828bec2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:13.319630 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-14f9924c-65e8-4d8f-91d0-1988c828bec2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:13.323306 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1051/2132] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:08:12 2022] DELETE /identity/v3/projects/88e506f9b86546ef8ba6dff6bccc9d00 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:08:13.328053 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa6ac1-9ffe-4e29-8dc8-79d82f654b18 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:13.328633 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa6ac1-9ffe-4e29-8dc8-79d82f654b18 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:13.328633 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa6ac1-9ffe-4e29-8dc8-79d82f654b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:13.351304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-caaa6ac1-9ffe-4e29-8dc8-79d82f654b18 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:08:13.357616 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-caaa6ac1-9ffe-4e29-8dc8-79d82f654b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:13.482041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1082/2133] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:08:13.488068 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30106559-c421-44a7-96f8-ba88ad4c4536 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:13.488353 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30106559-c421-44a7-96f8-ba88ad4c4536 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:13.488885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30106559-c421-44a7-96f8-ba88ad4c4536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:13.516204 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-30106559-c421-44a7-96f8-ba88ad4c4536 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:08:13.517463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-30106559-c421-44a7-96f8-ba88ad4c4536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:13.526632 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1052/2134] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:08:13.531125 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa6ac1-9ffe-4e29-8dc8-79d82f654b18 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:13.542383 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa6ac1-9ffe-4e29-8dc8-79d82f654b18 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:13.547328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37d960-bccc-4e6b-ac0b-0df8f06037d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:13.547962 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37d960-bccc-4e6b-ac0b-0df8f06037d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:13.548248 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37d960-bccc-4e6b-ac0b-0df8f06037d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:13.548533 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a37d960-bccc-4e6b-ac0b-0df8f06037d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:13.549245 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37d960-bccc-4e6b-ac0b-0df8f06037d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:08:13.549556 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37d960-bccc-4e6b-ac0b-0df8f06037d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:13.551424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a37d960-bccc-4e6b-ac0b-0df8f06037d4 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:13.564728 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1083/2135] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:08:13.569069 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30106559-c421-44a7-96f8-ba88ad4c4536 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:13.579563 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30106559-c421-44a7-96f8-ba88ad4c4536 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:13.585509 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0de2ce-e9bd-47ae-a244-345b3ed8ec6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:13.586545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0de2ce-e9bd-47ae-a244-345b3ed8ec6d admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:13.586545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0de2ce-e9bd-47ae-a244-345b3ed8ec6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:13.586739 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0de2ce-e9bd-47ae-a244-345b3ed8ec6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:13.587384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0de2ce-e9bd-47ae-a244-345b3ed8ec6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:13.589401 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0de2ce-e9bd-47ae-a244-345b3ed8ec6d admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:13.674681 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1053/2136] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:08:13 2022] POST /identity/v3/projects => generated 357 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:08:13.679460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37d960-bccc-4e6b-ac0b-0df8f06037d4 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:13.690444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a37d960-bccc-4e6b-ac0b-0df8f06037d4 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:13.754451 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807db0e-2830-4a8b-b094-ef6b47baeccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:13.754985 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807db0e-2830-4a8b-b094-ef6b47baeccb admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:13.755579 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807db0e-2830-4a8b-b094-ef6b47baeccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:13.755740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807db0e-2830-4a8b-b094-ef6b47baeccb admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:13.756331 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807db0e-2830-4a8b-b094-ef6b47baeccb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:13.758689 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807db0e-2830-4a8b-b094-ef6b47baeccb admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:13.847341 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1084/2137] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:08:13 2022] POST /identity/v3/users => generated 279 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:08:13.848729 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0de2ce-e9bd-47ae-a244-345b3ed8ec6d admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:13.860741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0de2ce-e9bd-47ae-a244-345b3ed8ec6d admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:13.866530 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb370b-9129-40b2-b37d-3952aa49a217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:13.867774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb370b-9129-40b2-b37d-3952aa49a217 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:13.867774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb370b-9129-40b2-b37d-3952aa49a217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:13.867774 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb370b-9129-40b2-b37d-3952aa49a217 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:13.868417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb370b-9129-40b2-b37d-3952aa49a217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:13.870178 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb370b-9129-40b2-b37d-3952aa49a217 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:13.880211 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:08:13.889344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1054/2138] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:08:13.890954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807db0e-2830-4a8b-b094-ef6b47baeccb admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:13.902522 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807db0e-2830-4a8b-b094-ef6b47baeccb admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:13.907547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c8fee-e472-4d52-bce6-d29e3b427f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:13.908182 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c8fee-e472-4d52-bce6-d29e3b427f07 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:13.908466 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c8fee-e472-4d52-bce6-d29e3b427f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:13.908742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c8fee-e472-4d52-bce6-d29e3b427f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:13.909476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6c8fee-e472-4d52-bce6-d29e3b427f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:13.911137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6c8fee-e472-4d52-bce6-d29e3b427f07 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:13.923369 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1085/2139] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:08:13.927238 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb370b-9129-40b2-b37d-3952aa49a217 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:13.938151 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb370b-9129-40b2-b37d-3952aa49a217 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:13.943495 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ed257-f5ad-4e59-b6ee-bce9141034be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:13.944182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ed257-f5ad-4e59-b6ee-bce9141034be admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:13.944777 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ed257-f5ad-4e59-b6ee-bce9141034be admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:13.945057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ed257-f5ad-4e59-b6ee-bce9141034be admin admin] PATH_INFO: `/v3/projects/e8e50690fdf84106b89e5cef78bbf8d8/users/ab472ee606c2414f8f578a3df7e78fae/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:13.962240 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ed257-f5ad-4e59-b6ee-bce9141034be admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8e50690fdf84106b89e5cef78bbf8d8, user_id=ab472ee606c2414f8f578a3df7e78fae, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:13.966420 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ed257-f5ad-4e59-b6ee-bce9141034be admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:14.090372 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1055/2140] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:08:13 2022] PUT /identity/v3/projects/e8e50690fdf84106b89e5cef78bbf8d8/users/ab472ee606c2414f8f578a3df7e78fae/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:08:14.102957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59073f89-64cb-4e19-b7ae-2216005a2cfb None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:14.103328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59073f89-64cb-4e19-b7ae-2216005a2cfb None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:14.103568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59073f89-64cb-4e19-b7ae-2216005a2cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:14.134147 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-59073f89-64cb-4e19-b7ae-2216005a2cfb None None] MFA Rules not processed for user `ab472ee606c2414f8f578a3df7e78fae`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:08:14.135423 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-59073f89-64cb-4e19-b7ae-2216005a2cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:14.165166 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:08:14.214051 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1086/2141] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:08:14 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:08:14.218833 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ed257-f5ad-4e59-b6ee-bce9141034be admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:14.283654 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ed257-f5ad-4e59-b6ee-bce9141034be admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:14.288489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b21188-bb35-4936-8b4f-d78865bd60b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:14.289125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b21188-bb35-4936-8b4f-d78865bd60b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:14.289415 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b21188-bb35-4936-8b4f-d78865bd60b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:14.289697 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b21188-bb35-4936-8b4f-d78865bd60b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:14.290408 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b21188-bb35-4936-8b4f-d78865bd60b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:14.292380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b21188-bb35-4936-8b4f-d78865bd60b5 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:14.408823 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1056/2142] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:08:14 2022] POST /identity/v3/projects => generated 355 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:08:14.411147 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59073f89-64cb-4e19-b7ae-2216005a2cfb None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:14.422941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59073f89-64cb-4e19-b7ae-2216005a2cfb None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:14.487258 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc57be-77a1-4099-8e15-6ac7476fd3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:14.488329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc57be-77a1-4099-8e15-6ac7476fd3ca admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:14.488776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc57be-77a1-4099-8e15-6ac7476fd3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:14.489387 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc57be-77a1-4099-8e15-6ac7476fd3ca admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:14.490118 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc57be-77a1-4099-8e15-6ac7476fd3ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:14.498313 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc57be-77a1-4099-8e15-6ac7476fd3ca admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:14.601878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1087/2143] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:08:14 2022] POST /identity/v3/users => generated 284 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:08:14.605999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b21188-bb35-4936-8b4f-d78865bd60b5 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:14.616915 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b21188-bb35-4936-8b4f-d78865bd60b5 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:14.622858 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b6fd9-9dba-4759-9779-75ffa163719d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:14.623251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b6fd9-9dba-4759-9779-75ffa163719d admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:14.623756 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b6fd9-9dba-4759-9779-75ffa163719d admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:14.623866 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b6fd9-9dba-4759-9779-75ffa163719d admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:14.624513 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b6fd9-9dba-4759-9779-75ffa163719d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:14.626278 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b6fd9-9dba-4759-9779-75ffa163719d admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:14.641106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1057/2144] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:08:14.646180 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc57be-77a1-4099-8e15-6ac7476fd3ca admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:14.692371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc57be-77a1-4099-8e15-6ac7476fd3ca admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:14.700363 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f9770-03fc-4816-ade1-79c2f984cd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:14.701435 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f9770-03fc-4816-ade1-79c2f984cd9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:14.701915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f9770-03fc-4816-ade1-79c2f984cd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:14.702381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44f9770-03fc-4816-ade1-79c2f984cd9d admin admin] PATH_INFO: `/v3/projects/b127130a704f4baa8a31e7bc36123cfa/users/be4d933961e749bfbde4b2b6234669e5/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:14.730653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f9770-03fc-4816-ade1-79c2f984cd9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b127130a704f4baa8a31e7bc36123cfa, user_id=be4d933961e749bfbde4b2b6234669e5, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:14.735601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44f9770-03fc-4816-ade1-79c2f984cd9d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:14.827414 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1088/2145] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:08:14 2022] PUT /identity/v3/projects/b127130a704f4baa8a31e7bc36123cfa/users/be4d933961e749bfbde4b2b6234669e5/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:08:14.828536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b6fd9-9dba-4759-9779-75ffa163719d admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:14.839462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b6fd9-9dba-4759-9779-75ffa163719d admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:14.904325 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8482f-232f-4dc5-8549-b06cb1872118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:14.905247 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8482f-232f-4dc5-8549-b06cb1872118 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:14.905534 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8482f-232f-4dc5-8549-b06cb1872118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:14.905941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8482f-232f-4dc5-8549-b06cb1872118 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:14.906557 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8482f-232f-4dc5-8549-b06cb1872118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:14.908203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda8482f-232f-4dc5-8549-b06cb1872118 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:14.919759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1058/2146] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:08:14.924988 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f9770-03fc-4816-ade1-79c2f984cd9d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:14.943964 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44f9770-03fc-4816-ade1-79c2f984cd9d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:14.951050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d3128-3955-45b3-9ae8-e63755b328da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:14.951914 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d3128-3955-45b3-9ae8-e63755b328da admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:14.952266 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d3128-3955-45b3-9ae8-e63755b328da admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:14.952615 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d3128-3955-45b3-9ae8-e63755b328da admin admin] PATH_INFO: `/v3/projects/b127130a704f4baa8a31e7bc36123cfa/users/be4d933961e749bfbde4b2b6234669e5/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:14.954763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d3128-3955-45b3-9ae8-e63755b328da admin admin] RBAC: Authorizing `identity:create_grant(project_id=b127130a704f4baa8a31e7bc36123cfa, user_id=be4d933961e749bfbde4b2b6234669e5, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:14.957756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d3128-3955-45b3-9ae8-e63755b328da admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:14.970628 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1089/2147] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:08:14 2022] PUT /identity/v3/projects/b127130a704f4baa8a31e7bc36123cfa/users/be4d933961e749bfbde4b2b6234669e5/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:08:14.985765 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21bd127-efa5-4d1e-b898-fbfd07e63265 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:14.986359 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21bd127-efa5-4d1e-b898-fbfd07e63265 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:14.986359 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21bd127-efa5-4d1e-b898-fbfd07e63265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:15.018047 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-a21bd127-efa5-4d1e-b898-fbfd07e63265 None None] MFA Rules not processed for user `be4d933961e749bfbde4b2b6234669e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:08:15.019282 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-a21bd127-efa5-4d1e-b898-fbfd07e63265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:15.055057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:08:15.097380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1059/2148] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:08:14 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:08:15.108800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d3128-3955-45b3-9ae8-e63755b328da admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:15.110611 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-182d3128-3955-45b3-9ae8-e63755b328da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:15.176348 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d3128-3955-45b3-9ae8-e63755b328da admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:15.181312 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a8d19-06f6-4677-b778-d0dddd86048c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:15.182175 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a8d19-06f6-4677-b778-d0dddd86048c service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:15.182507 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a8d19-06f6-4677-b778-d0dddd86048c service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:15.182749 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5a8d19-06f6-4677-b778-d0dddd86048c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:15.194463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a8d19-06f6-4677-b778-d0dddd86048c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:15.195968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5a8d19-06f6-4677-b778-d0dddd86048c service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:15.206868 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1090/2149] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:08:15 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:08:15.461996 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21bd127-efa5-4d1e-b898-fbfd07e63265 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:15.472134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21bd127-efa5-4d1e-b898-fbfd07e63265 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:15.475865 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4574ff92-5830-4a01-8823-ba3e0bc59e4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:15.476619 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574ff92-5830-4a01-8823-ba3e0bc59e4a service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:15.477642 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574ff92-5830-4a01-8823-ba3e0bc59e4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:15.477963 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4574ff92-5830-4a01-8823-ba3e0bc59e4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:15.538734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574ff92-5830-4a01-8823-ba3e0bc59e4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:15.540786 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4574ff92-5830-4a01-8823-ba3e0bc59e4a service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:15.550295 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1060/2150] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:08:15 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:08:15.736539 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a8d19-06f6-4677-b778-d0dddd86048c service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:15.738158 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4c5a8d19-06f6-4677-b778-d0dddd86048c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:15.749408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5a8d19-06f6-4677-b778-d0dddd86048c service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:15.754167 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae84506-c046-478d-9c17-238e5258ada3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:15.754804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae84506-c046-478d-9c17-238e5258ada3 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:15.755068 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae84506-c046-478d-9c17-238e5258ada3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:15.755345 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae84506-c046-478d-9c17-238e5258ada3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:15.757249 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0ae84506-c046-478d-9c17-238e5258ada3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:15.780841 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:08:15.846664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae84506-c046-478d-9c17-238e5258ada3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:15.849690 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae84506-c046-478d-9c17-238e5258ada3 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:15.863403 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1091/2151] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:08:15 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:08:23.484122 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0bcc64-c1df-465d-ba9b-c96240b569cb None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:23.484603 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0bcc64-c1df-465d-ba9b-c96240b569cb None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:23.484693 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0bcc64-c1df-465d-ba9b-c96240b569cb None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:23.486308 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1061/2152] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 2 09:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:08:23.517569 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005745fd-ca37-4648-b11f-eed26c0d7827 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:23.518168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005745fd-ca37-4648-b11f-eed26c0d7827 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:23.518310 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005745fd-ca37-4648-b11f-eed26c0d7827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:23.540231 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-005745fd-ca37-4648-b11f-eed26c0d7827 None None] MFA Rules not processed for user `cad5cc9927374836b0cc74db85b686fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:08:23.541453 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-005745fd-ca37-4648-b11f-eed26c0d7827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:23.607639 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1092/2153] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 2 09:08:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:08:23.619006 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0bcc64-c1df-465d-ba9b-c96240b569cb None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:23.627526 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-2e0bcc64-c1df-465d-ba9b-c96240b569cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:23.645389 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0bcc64-c1df-465d-ba9b-c96240b569cb None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:23.652199 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf40c04-f31e-415a-b160-f9432c738f4a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad5cc9927374836b0cc74db85b686fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:23.653497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf40c04-f31e-415a-b160-f9432c738f4a service placement] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:23.653497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf40c04-f31e-415a-b160-f9432c738f4a service placement] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:23.654170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf40c04-f31e-415a-b160-f9432c738f4a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:23.665262 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf40c04-f31e-415a-b160-f9432c738f4a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:23.666475 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf40c04-f31e-415a-b160-f9432c738f4a service placement] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:23.685064 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1062/2154] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:08:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:08:24.421934 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005745fd-ca37-4648-b11f-eed26c0d7827 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:24.423578 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-005745fd-ca37-4648-b11f-eed26c0d7827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:24.495115 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005745fd-ca37-4648-b11f-eed26c0d7827 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:24.503140 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdbba5-dc1c-4409-985d-2b5823293ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:24.503955 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdbba5-dc1c-4409-985d-2b5823293ec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:24.504341 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdbba5-dc1c-4409-985d-2b5823293ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:24.504722 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdbba5-dc1c-4409-985d-2b5823293ec6 admin admin] PATH_INFO: `/v3/users/8e029b3a928940a1bc7e6ac883595069` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:24.508134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efdbba5-dc1c-4409-985d-2b5823293ec6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e029b3a928940a1bc7e6ac883595069)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:24.508456 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efdbba5-dc1c-4409-985d-2b5823293ec6 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:24.807924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0efdbba5-dc1c-4409-985d-2b5823293ec6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e029b3a928940a1bc7e6ac883595069'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:24.813035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:08:24.816756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0efdbba5-dc1c-4409-985d-2b5823293ec6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e029b3a928940a1bc7e6ac883595069'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:24.817790 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0efdbba5-dc1c-4409-985d-2b5823293ec6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e029b3a928940a1bc7e6ac883595069'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:24.831906 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0efdbba5-dc1c-4409-985d-2b5823293ec6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e029b3a928940a1bc7e6ac883595069'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:24.832766 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0efdbba5-dc1c-4409-985d-2b5823293ec6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e029b3a928940a1bc7e6ac883595069'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:24.949856 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1093/2155] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:08:24 2022] DELETE /identity/v3/users/8e029b3a928940a1bc7e6ac883595069 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:08:25.452518 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf40c04-f31e-415a-b160-f9432c738f4a service placement] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:25.466692 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf40c04-f31e-415a-b160-f9432c738f4a service placement] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:25.553428 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42619ab5-e146-458e-8a03-3afcdb616561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:25.554440 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42619ab5-e146-458e-8a03-3afcdb616561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:25.555222 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42619ab5-e146-458e-8a03-3afcdb616561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:25.555506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42619ab5-e146-458e-8a03-3afcdb616561 admin admin] PATH_INFO: `/v3/projects/9d3d987d8b484c639b4c121b63e7657b` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:25.556860 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42619ab5-e146-458e-8a03-3afcdb616561 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d3d987d8b484c639b4c121b63e7657b)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:25.560104 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42619ab5-e146-458e-8a03-3afcdb616561 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:25.757729 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-42619ab5-e146-458e-8a03-3afcdb616561 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d3d987d8b484c639b4c121b63e7657b', 'request_id': 'req-42619ab5-e146-458e-8a03-3afcdb616561'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:25.764467 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-42619ab5-e146-458e-8a03-3afcdb616561 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d3d987d8b484c639b4c121b63e7657b', 'request_id': 'req-42619ab5-e146-458e-8a03-3afcdb616561'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:25.771414 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:08:25.811128 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-42619ab5-e146-458e-8a03-3afcdb616561 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d3d987d8b484c639b4c121b63e7657b', 'request_id': 'req-42619ab5-e146-458e-8a03-3afcdb616561'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:25.895601 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-42619ab5-e146-458e-8a03-3afcdb616561 admin admin] The token cache is being invalidate because project 9d3d987d8b484c639b4c121b63e7657b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:08:25.895845 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-42619ab5-e146-458e-8a03-3afcdb616561 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:25.896551 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-42619ab5-e146-458e-8a03-3afcdb616561 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:25.899060 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1063/2156] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:08:25 2022] DELETE /identity/v3/projects/9d3d987d8b484c639b4c121b63e7657b => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:08:25.903303 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdbba5-dc1c-4409-985d-2b5823293ec6 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:25.905098 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0efdbba5-dc1c-4409-985d-2b5823293ec6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:25.920802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdbba5-dc1c-4409-985d-2b5823293ec6 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:26.008326 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e5186f-59eb-4fb5-ba34-a5acffccca0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:26.008927 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5186f-59eb-4fb5-ba34-a5acffccca0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:26.009392 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5186f-59eb-4fb5-ba34-a5acffccca0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:26.009682 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5186f-59eb-4fb5-ba34-a5acffccca0b admin admin] PATH_INFO: `/v3/users/e59092ffb4374aaa8800069c8b1fdf18` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:26.010967 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5186f-59eb-4fb5-ba34-a5acffccca0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e59092ffb4374aaa8800069c8b1fdf18)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:26.012965 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e5186f-59eb-4fb5-ba34-a5acffccca0b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:26.257337 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-59e5186f-59eb-4fb5-ba34-a5acffccca0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e59092ffb4374aaa8800069c8b1fdf18'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:26.264924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-59e5186f-59eb-4fb5-ba34-a5acffccca0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e59092ffb4374aaa8800069c8b1fdf18'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:26.265526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-59e5186f-59eb-4fb5-ba34-a5acffccca0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e59092ffb4374aaa8800069c8b1fdf18'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:26.279627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-59e5186f-59eb-4fb5-ba34-a5acffccca0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e59092ffb4374aaa8800069c8b1fdf18'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:26.280406 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-59e5186f-59eb-4fb5-ba34-a5acffccca0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e59092ffb4374aaa8800069c8b1fdf18'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:26.357784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1094/2157] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:08:25 2022] DELETE /identity/v3/users/e59092ffb4374aaa8800069c8b1fdf18 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:08:26.965258 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42619ab5-e146-458e-8a03-3afcdb616561 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:26.966862 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-42619ab5-e146-458e-8a03-3afcdb616561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:27.000597 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42619ab5-e146-458e-8a03-3afcdb616561 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:27.070113 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79451366-2435-4e6f-b9af-6fa92cc4cb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:27.070859 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79451366-2435-4e6f-b9af-6fa92cc4cb1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:27.071210 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79451366-2435-4e6f-b9af-6fa92cc4cb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:27.071561 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79451366-2435-4e6f-b9af-6fa92cc4cb1c admin admin] PATH_INFO: `/v3/projects/6648f2058e6b4b93943cf95756c9a391` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:27.073363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79451366-2435-4e6f-b9af-6fa92cc4cb1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6648f2058e6b4b93943cf95756c9a391)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:27.075594 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79451366-2435-4e6f-b9af-6fa92cc4cb1c admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:27.294553 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-79451366-2435-4e6f-b9af-6fa92cc4cb1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6648f2058e6b4b93943cf95756c9a391', 'request_id': 'req-79451366-2435-4e6f-b9af-6fa92cc4cb1c'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:27.300878 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-79451366-2435-4e6f-b9af-6fa92cc4cb1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6648f2058e6b4b93943cf95756c9a391', 'request_id': 'req-79451366-2435-4e6f-b9af-6fa92cc4cb1c'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:27.306726 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-79451366-2435-4e6f-b9af-6fa92cc4cb1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6648f2058e6b4b93943cf95756c9a391', 'request_id': 'req-79451366-2435-4e6f-b9af-6fa92cc4cb1c'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:27.640968 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-79451366-2435-4e6f-b9af-6fa92cc4cb1c admin admin] The token cache is being invalidate because project 6648f2058e6b4b93943cf95756c9a391 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:08:27.641244 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-79451366-2435-4e6f-b9af-6fa92cc4cb1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:27.641878 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-79451366-2435-4e6f-b9af-6fa92cc4cb1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:08:27.643658 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1064/2158] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:08:26 2022] DELETE /identity/v3/projects/6648f2058e6b4b93943cf95756c9a391 => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:08:27.648834 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70729d5-d8b3-473a-bb73-61a302b16901 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:27.649288 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70729d5-d8b3-473a-bb73-61a302b16901 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:27.649467 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70729d5-d8b3-473a-bb73-61a302b16901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:27.672515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f70729d5-d8b3-473a-bb73-61a302b16901 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:08:27.674040 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f70729d5-d8b3-473a-bb73-61a302b16901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:27.725828 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:08:27.740447 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1095/2159] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:08:27.749242 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070d6c0-004a-4810-b9c2-6cf970e4d373 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:27.749527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070d6c0-004a-4810-b9c2-6cf970e4d373 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:27.749804 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070d6c0-004a-4810-b9c2-6cf970e4d373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:27.770387 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-c070d6c0-004a-4810-b9c2-6cf970e4d373 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:08:27.771614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-c070d6c0-004a-4810-b9c2-6cf970e4d373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:27.781179 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1065/2160] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:08:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:08:27.793525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70729d5-d8b3-473a-bb73-61a302b16901 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:27.804053 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70729d5-d8b3-473a-bb73-61a302b16901 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:27.812004 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9a3b3-c0ca-4a42-a44f-ff196800753d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:27.812640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9a3b3-c0ca-4a42-a44f-ff196800753d admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:27.812928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9a3b3-c0ca-4a42-a44f-ff196800753d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:27.813203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b9a3b3-c0ca-4a42-a44f-ff196800753d admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:27.813933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9a3b3-c0ca-4a42-a44f-ff196800753d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:08:27.814244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9a3b3-c0ca-4a42-a44f-ff196800753d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:27.815921 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b9a3b3-c0ca-4a42-a44f-ff196800753d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:27.829096 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1096/2161] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:08:27.835052 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070d6c0-004a-4810-b9c2-6cf970e4d373 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:27.845187 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070d6c0-004a-4810-b9c2-6cf970e4d373 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:27.850259 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dffe8d-391b-4e7e-9f23-9de30e8c8c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:27.850879 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dffe8d-391b-4e7e-9f23-9de30e8c8c8a admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:27.852033 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dffe8d-391b-4e7e-9f23-9de30e8c8c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:27.852033 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dffe8d-391b-4e7e-9f23-9de30e8c8c8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:27.852227 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dffe8d-391b-4e7e-9f23-9de30e8c8c8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:27.854093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dffe8d-391b-4e7e-9f23-9de30e8c8c8a admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:28.088789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1066/2162] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:08:27 2022] POST /identity/v3/projects => generated 377 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:08:28.093020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9a3b3-c0ca-4a42-a44f-ff196800753d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:28.103152 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b9a3b3-c0ca-4a42-a44f-ff196800753d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:28.163230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0de3ac-aa1a-4df0-ab54-23b277bf659f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:28.163933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0de3ac-aa1a-4df0-ab54-23b277bf659f admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:28.164244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0de3ac-aa1a-4df0-ab54-23b277bf659f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:28.164521 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0de3ac-aa1a-4df0-ab54-23b277bf659f admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:28.165183 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0de3ac-aa1a-4df0-ab54-23b277bf659f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:28.167202 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0de3ac-aa1a-4df0-ab54-23b277bf659f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:28.361699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1097/2163] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:08:28 2022] POST /identity/v3/users => generated 289 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:08:28.363183 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dffe8d-391b-4e7e-9f23-9de30e8c8c8a admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:28.373936 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dffe8d-391b-4e7e-9f23-9de30e8c8c8a admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:28.378424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4093f8-4cf7-4c47-8302-f65748a610d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:28.379021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4093f8-4cf7-4c47-8302-f65748a610d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:28.379299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4093f8-4cf7-4c47-8302-f65748a610d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:28.379575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4093f8-4cf7-4c47-8302-f65748a610d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:28.380257 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4093f8-4cf7-4c47-8302-f65748a610d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:28.381856 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4093f8-4cf7-4c47-8302-f65748a610d9 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:28.392506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1067/2164] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:08:28.396602 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0de3ac-aa1a-4df0-ab54-23b277bf659f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:28.407168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0de3ac-aa1a-4df0-ab54-23b277bf659f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:28.412281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947b179-06aa-4063-a495-c4bd148c54d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:28.412935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947b179-06aa-4063-a495-c4bd148c54d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:28.413257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947b179-06aa-4063-a495-c4bd148c54d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:28.413537 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947b179-06aa-4063-a495-c4bd148c54d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:28.414805 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947b179-06aa-4063-a495-c4bd148c54d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:28.416496 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947b179-06aa-4063-a495-c4bd148c54d6 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:28.427956 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1098/2165] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:08:28.432703 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4093f8-4cf7-4c47-8302-f65748a610d9 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:28.442550 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4093f8-4cf7-4c47-8302-f65748a610d9 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:28.447299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d3b40-da8c-47d0-a562-e6318db01658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:28.452831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d3b40-da8c-47d0-a562-e6318db01658 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:28.452831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d3b40-da8c-47d0-a562-e6318db01658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:28.452831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d3b40-da8c-47d0-a562-e6318db01658 admin admin] PATH_INFO: `/v3/projects/5763b774e5814ca38900cfb8bee110a0/users/2c6f9cb461cc480dbc84496dfd3e2b5c/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:28.463755 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d3b40-da8c-47d0-a562-e6318db01658 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5763b774e5814ca38900cfb8bee110a0, user_id=2c6f9cb461cc480dbc84496dfd3e2b5c, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:28.466566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d3b40-da8c-47d0-a562-e6318db01658 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:28.596864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1068/2166] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:08:28 2022] PUT /identity/v3/projects/5763b774e5814ca38900cfb8bee110a0/users/2c6f9cb461cc480dbc84496dfd3e2b5c/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:08:28.623462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1cd40-1572-4a81-94a6-5bb95ce990f1 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:28.623828 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1cd40-1572-4a81-94a6-5bb95ce990f1 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:28.624178 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1cd40-1572-4a81-94a6-5bb95ce990f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:28.651266 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-29d1cd40-1572-4a81-94a6-5bb95ce990f1 None None] MFA Rules not processed for user `2c6f9cb461cc480dbc84496dfd3e2b5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:08:28.652491 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-29d1cd40-1572-4a81-94a6-5bb95ce990f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:28.720094 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1099/2167] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:08:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:08:28.732872 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d3b40-da8c-47d0-a562-e6318db01658 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:08:28.734787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-1c4d3b40-da8c-47d0-a562-e6318db01658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:08:28.750799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:08:28.796344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d3b40-da8c-47d0-a562-e6318db01658 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:08:28.800284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96437e70-6169-42be-bf1e-18c719610703 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:08:28.800910 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96437e70-6169-42be-bf1e-18c719610703 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:08:28.801194 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96437e70-6169-42be-bf1e-18c719610703 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:08:28.801472 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96437e70-6169-42be-bf1e-18c719610703 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:08:28.810631 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96437e70-6169-42be-bf1e-18c719610703 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:08:28.812063 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96437e70-6169-42be-bf1e-18c719610703 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:08:28.821072 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1069/2168] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:08:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:08:39.472209 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=77,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:08:40.024362 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=20 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:05.982860 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1cd40-1572-4a81-94a6-5bb95ce990f1 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:05.985334 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-29d1cd40-1572-4a81-94a6-5bb95ce990f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:06.064085 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d1cd40-1572-4a81-94a6-5bb95ce990f1 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:06.069196 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b24589-4a9d-42be-9b1b-d8d18a415709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:06.069920 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b24589-4a9d-42be-9b1b-d8d18a415709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:06.070276 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b24589-4a9d-42be-9b1b-d8d18a415709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:06.070601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b24589-4a9d-42be-9b1b-d8d18a415709 admin admin] PATH_INFO: `/v3/users/ab472ee606c2414f8f578a3df7e78fae` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:06.071948 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b24589-4a9d-42be-9b1b-d8d18a415709 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab472ee606c2414f8f578a3df7e78fae)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:06.074076 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b24589-4a9d-42be-9b1b-d8d18a415709 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:06.310363 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-41b24589-4a9d-42be-9b1b-d8d18a415709 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab472ee606c2414f8f578a3df7e78fae'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:06.318475 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-41b24589-4a9d-42be-9b1b-d8d18a415709 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab472ee606c2414f8f578a3df7e78fae'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:06.319386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-41b24589-4a9d-42be-9b1b-d8d18a415709 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab472ee606c2414f8f578a3df7e78fae'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:06.340117 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-41b24589-4a9d-42be-9b1b-d8d18a415709 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab472ee606c2414f8f578a3df7e78fae'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:06.341290 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-41b24589-4a9d-42be-9b1b-d8d18a415709 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab472ee606c2414f8f578a3df7e78fae'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:06.418632 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1100/2169] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:09:05 2022] DELETE /identity/v3/users/ab472ee606c2414f8f578a3df7e78fae => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:09:06.423913 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7a742-a847-4a99-945d-057d6bda8474 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:06.425145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7a742-a847-4a99-945d-057d6bda8474 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:06.425583 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7a742-a847-4a99-945d-057d6bda8474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:06.454201 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-34a7a742-a847-4a99-945d-057d6bda8474 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:06.456040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-34a7a742-a847-4a99-945d-057d6bda8474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:06.542028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1070/2170] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:06.552344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b24589-4a9d-42be-9b1b-d8d18a415709 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:06.555025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-41b24589-4a9d-42be-9b1b-d8d18a415709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:06.620780 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b24589-4a9d-42be-9b1b-d8d18a415709 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:06.624798 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f66c22-dff4-47ef-a88a-d727da80a91c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:06.625425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f66c22-dff4-47ef-a88a-d727da80a91c service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:06.625708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f66c22-dff4-47ef-a88a-d727da80a91c service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:06.626026 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f66c22-dff4-47ef-a88a-d727da80a91c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:06.637081 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f66c22-dff4-47ef-a88a-d727da80a91c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:06.639433 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f66c22-dff4-47ef-a88a-d727da80a91c service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:06.651132 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1101/2171] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:09:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:09:07.061605 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7a742-a847-4a99-945d-057d6bda8474 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:07.063990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-34a7a742-a847-4a99-945d-057d6bda8474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:07.081366 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7a742-a847-4a99-945d-057d6bda8474 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:07.086807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a32f3-9146-4336-9f1b-78f5e99f5590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:07.087458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a32f3-9146-4336-9f1b-78f5e99f5590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:07.087740 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a32f3-9146-4336-9f1b-78f5e99f5590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:07.088012 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a32f3-9146-4336-9f1b-78f5e99f5590 admin admin] PATH_INFO: `/v3/projects/e8e50690fdf84106b89e5cef78bbf8d8` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:07.089180 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a32f3-9146-4336-9f1b-78f5e99f5590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8e50690fdf84106b89e5cef78bbf8d8)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:07.091174 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a32f3-9146-4336-9f1b-78f5e99f5590 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:07.203770 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e39a32f3-9146-4336-9f1b-78f5e99f5590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8e50690fdf84106b89e5cef78bbf8d8', 'request_id': 'req-e39a32f3-9146-4336-9f1b-78f5e99f5590'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:07.210735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e39a32f3-9146-4336-9f1b-78f5e99f5590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8e50690fdf84106b89e5cef78bbf8d8', 'request_id': 'req-e39a32f3-9146-4336-9f1b-78f5e99f5590'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:07.216445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e39a32f3-9146-4336-9f1b-78f5e99f5590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8e50690fdf84106b89e5cef78bbf8d8', 'request_id': 'req-e39a32f3-9146-4336-9f1b-78f5e99f5590'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:07.292234 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e39a32f3-9146-4336-9f1b-78f5e99f5590 admin admin] The token cache is being invalidate because project e8e50690fdf84106b89e5cef78bbf8d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:09:07.292521 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e39a32f3-9146-4336-9f1b-78f5e99f5590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:07.293180 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e39a32f3-9146-4336-9f1b-78f5e99f5590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:07.294726 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1071/2172] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:09:07 2022] DELETE /identity/v3/projects/e8e50690fdf84106b89e5cef78bbf8d8 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:09:07.298773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f66c22-dff4-47ef-a88a-d727da80a91c service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:07.309351 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f66c22-dff4-47ef-a88a-d727da80a91c service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:07.329560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:07.369091 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ca588-3cfd-404b-893d-759000883021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:07.370041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ca588-3cfd-404b-893d-759000883021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:07.370328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ca588-3cfd-404b-893d-759000883021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:07.370610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ca588-3cfd-404b-893d-759000883021 admin admin] PATH_INFO: `/v3/users/be4d933961e749bfbde4b2b6234669e5` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:07.371826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ca588-3cfd-404b-893d-759000883021 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be4d933961e749bfbde4b2b6234669e5)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:07.373755 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ca588-3cfd-404b-893d-759000883021 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:07.603073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-332ca588-3cfd-404b-893d-759000883021 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be4d933961e749bfbde4b2b6234669e5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:07.610505 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-332ca588-3cfd-404b-893d-759000883021 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be4d933961e749bfbde4b2b6234669e5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:07.611302 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-332ca588-3cfd-404b-893d-759000883021 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be4d933961e749bfbde4b2b6234669e5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:07.625551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-332ca588-3cfd-404b-893d-759000883021 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be4d933961e749bfbde4b2b6234669e5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:07.626341 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-332ca588-3cfd-404b-893d-759000883021 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be4d933961e749bfbde4b2b6234669e5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:07.749737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1102/2173] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:09:07 2022] DELETE /identity/v3/users/be4d933961e749bfbde4b2b6234669e5 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:09:08.448133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a32f3-9146-4336-9f1b-78f5e99f5590 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:08.449645 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e39a32f3-9146-4336-9f1b-78f5e99f5590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:08.464062 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a32f3-9146-4336-9f1b-78f5e99f5590 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:08.527317 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b38e1-97cc-4f92-94ef-dbb2ce73ca48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:08.528091 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b38e1-97cc-4f92-94ef-dbb2ce73ca48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:08.528424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b38e1-97cc-4f92-94ef-dbb2ce73ca48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:08.528714 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b38e1-97cc-4f92-94ef-dbb2ce73ca48 admin admin] PATH_INFO: `/v3/projects/b127130a704f4baa8a31e7bc36123cfa` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:08.530122 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b38e1-97cc-4f92-94ef-dbb2ce73ca48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b127130a704f4baa8a31e7bc36123cfa)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:08.532107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b38e1-97cc-4f92-94ef-dbb2ce73ca48 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:08.625828 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:08.645662 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-ef5b38e1-97cc-4f92-94ef-dbb2ce73ca48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b127130a704f4baa8a31e7bc36123cfa', 'request_id': 'req-ef5b38e1-97cc-4f92-94ef-dbb2ce73ca48'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:08.652232 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-ef5b38e1-97cc-4f92-94ef-dbb2ce73ca48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b127130a704f4baa8a31e7bc36123cfa', 'request_id': 'req-ef5b38e1-97cc-4f92-94ef-dbb2ce73ca48'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:08.660480 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-ef5b38e1-97cc-4f92-94ef-dbb2ce73ca48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b127130a704f4baa8a31e7bc36123cfa', 'request_id': 'req-ef5b38e1-97cc-4f92-94ef-dbb2ce73ca48'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:08.724905 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-ef5b38e1-97cc-4f92-94ef-dbb2ce73ca48 admin admin] The token cache is being invalidate because project b127130a704f4baa8a31e7bc36123cfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:09:08.725407 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-ef5b38e1-97cc-4f92-94ef-dbb2ce73ca48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:08.726339 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-ef5b38e1-97cc-4f92-94ef-dbb2ce73ca48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:08.727954 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1072/2174] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:09:08 2022] DELETE /identity/v3/projects/b127130a704f4baa8a31e7bc36123cfa => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:09:08.735452 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47168222-981e-466d-b028-7817651e5f6f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:08.736297 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47168222-981e-466d-b028-7817651e5f6f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:08.736572 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47168222-981e-466d-b028-7817651e5f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:08.760253 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-47168222-981e-466d-b028-7817651e5f6f None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:08.761637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-47168222-981e-466d-b028-7817651e5f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:08.829364 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1103/2175] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:08.841770 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feaa69e-99f9-42bc-bf6a-9e7eee05814a None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:08.842334 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feaa69e-99f9-42bc-bf6a-9e7eee05814a None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:08.842639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feaa69e-99f9-42bc-bf6a-9e7eee05814a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:08.866489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-7feaa69e-99f9-42bc-bf6a-9e7eee05814a None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:08.867793 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-7feaa69e-99f9-42bc-bf6a-9e7eee05814a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:08.876483 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1073/2176] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:08.881482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47168222-981e-466d-b028-7817651e5f6f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:08.891587 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47168222-981e-466d-b028-7817651e5f6f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:08.896272 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7938bf-83fc-4de8-8917-b30d2a9da945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:08.896897 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7938bf-83fc-4de8-8917-b30d2a9da945 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:08.897187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7938bf-83fc-4de8-8917-b30d2a9da945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:08.897457 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7938bf-83fc-4de8-8917-b30d2a9da945 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:08.898206 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7938bf-83fc-4de8-8917-b30d2a9da945 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:09:08.898511 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7938bf-83fc-4de8-8917-b30d2a9da945 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:08.900118 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7938bf-83fc-4de8-8917-b30d2a9da945 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:08.913226 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1104/2177] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:09:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:09:08.917950 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feaa69e-99f9-42bc-bf6a-9e7eee05814a None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:08.928719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feaa69e-99f9-42bc-bf6a-9e7eee05814a None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:08.933645 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3144d028-1dee-4b6c-8e15-9746e2bb998f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:08.934571 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144d028-1dee-4b6c-8e15-9746e2bb998f admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:08.934970 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144d028-1dee-4b6c-8e15-9746e2bb998f admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:08.935355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3144d028-1dee-4b6c-8e15-9746e2bb998f admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:08.936411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3144d028-1dee-4b6c-8e15-9746e2bb998f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:08.939570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3144d028-1dee-4b6c-8e15-9746e2bb998f admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:09.074437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1074/2178] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:09:08 2022] POST /identity/v3/projects => generated 373 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:09:09.080817 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7938bf-83fc-4de8-8917-b30d2a9da945 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:09.092198 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7938bf-83fc-4de8-8917-b30d2a9da945 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:09.122699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:09.157601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763319b-9dc5-42db-a897-95a3d0649e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:09.158254 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763319b-9dc5-42db-a897-95a3d0649e64 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:09.158572 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763319b-9dc5-42db-a897-95a3d0649e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:09.158822 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763319b-9dc5-42db-a897-95a3d0649e64 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:09.159514 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763319b-9dc5-42db-a897-95a3d0649e64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:09.161463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763319b-9dc5-42db-a897-95a3d0649e64 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:09.754340 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1105/2179] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:09:09 2022] POST /identity/v3/users => generated 287 bytes in 674 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:09:09.759096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3144d028-1dee-4b6c-8e15-9746e2bb998f admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:09.770227 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3144d028-1dee-4b6c-8e15-9746e2bb998f admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:09.780079 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ecda5-697a-4ce5-83eb-ea0f5f7d755a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:09.780708 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ecda5-697a-4ce5-83eb-ea0f5f7d755a admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:09.781021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ecda5-697a-4ce5-83eb-ea0f5f7d755a admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:09.781292 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ecda5-697a-4ce5-83eb-ea0f5f7d755a admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:09.782056 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ecda5-697a-4ce5-83eb-ea0f5f7d755a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:09.783781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ecda5-697a-4ce5-83eb-ea0f5f7d755a admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:09.795967 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1075/2180] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:09:09.800944 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763319b-9dc5-42db-a897-95a3d0649e64 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:09.811971 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763319b-9dc5-42db-a897-95a3d0649e64 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:09.816810 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe08189-4abb-4325-9d48-6c197f01c3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:09.817437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe08189-4abb-4325-9d48-6c197f01c3d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:09.817724 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe08189-4abb-4325-9d48-6c197f01c3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:09.818040 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe08189-4abb-4325-9d48-6c197f01c3d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:09.819264 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe08189-4abb-4325-9d48-6c197f01c3d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:09.820916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe08189-4abb-4325-9d48-6c197f01c3d2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:09.832748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1106/2181] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:09:09.837457 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ecda5-697a-4ce5-83eb-ea0f5f7d755a admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:09.850429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ecda5-697a-4ce5-83eb-ea0f5f7d755a admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:09.856273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03099033-87ea-4823-b264-e9459b7030cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:09.856993 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03099033-87ea-4823-b264-e9459b7030cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:09.857269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03099033-87ea-4823-b264-e9459b7030cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:09.857543 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03099033-87ea-4823-b264-e9459b7030cc admin admin] PATH_INFO: `/v3/projects/2869c44d3e3d4d6aa69226f5ac3cf848/users/964a1cb3b1f043088f074f8986183bb3/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:09.878638 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03099033-87ea-4823-b264-e9459b7030cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2869c44d3e3d4d6aa69226f5ac3cf848, user_id=964a1cb3b1f043088f074f8986183bb3, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:09.881923 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03099033-87ea-4823-b264-e9459b7030cc admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:09.985406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1076/2182] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:09:09 2022] PUT /identity/v3/projects/2869c44d3e3d4d6aa69226f5ac3cf848/users/964a1cb3b1f043088f074f8986183bb3/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:09:10.002247 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4998a-a7fa-4817-a006-1d8edff1c909 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:10.002689 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4998a-a7fa-4817-a006-1d8edff1c909 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:10.003186 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4998a-a7fa-4817-a006-1d8edff1c909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:10.046388 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5af4998a-a7fa-4817-a006-1d8edff1c909 None None] MFA Rules not processed for user `964a1cb3b1f043088f074f8986183bb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:10.048333 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5af4998a-a7fa-4817-a006-1d8edff1c909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:10.137659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1107/2183] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:10.149183 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03099033-87ea-4823-b264-e9459b7030cc admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:10.151065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-03099033-87ea-4823-b264-e9459b7030cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:10.213336 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03099033-87ea-4823-b264-e9459b7030cc admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:10.217121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0afd2a5-7305-4265-8d1e-5a001dead0ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:10.217748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0afd2a5-7305-4265-8d1e-5a001dead0ef service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:10.218280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0afd2a5-7305-4265-8d1e-5a001dead0ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:10.218410 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0afd2a5-7305-4265-8d1e-5a001dead0ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:10.227620 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0afd2a5-7305-4265-8d1e-5a001dead0ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:10.229068 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0afd2a5-7305-4265-8d1e-5a001dead0ef service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:10.239016 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1077/2184] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:09:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:09:13.878263 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af4998a-a7fa-4817-a006-1d8edff1c909 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:13.879830 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5af4998a-a7fa-4817-a006-1d8edff1c909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:13.986621 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af4998a-a7fa-4817-a006-1d8edff1c909 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:13.992421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d3fdca-05b8-4902-8545-8c2d0a707242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:13.993281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3fdca-05b8-4902-8545-8c2d0a707242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:13.993554 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3fdca-05b8-4902-8545-8c2d0a707242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:13.993867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3fdca-05b8-4902-8545-8c2d0a707242 admin admin] PATH_INFO: `/v3/users/45d6d1b60eda44139a2dae7de22c97b4` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:13.995093 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d3fdca-05b8-4902-8545-8c2d0a707242 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45d6d1b60eda44139a2dae7de22c97b4)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:13.997113 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d3fdca-05b8-4902-8545-8c2d0a707242 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:14.213916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:14.226979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-18d3fdca-05b8-4902-8545-8c2d0a707242 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45d6d1b60eda44139a2dae7de22c97b4'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:14.239373 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-18d3fdca-05b8-4902-8545-8c2d0a707242 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45d6d1b60eda44139a2dae7de22c97b4'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:14.244560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-18d3fdca-05b8-4902-8545-8c2d0a707242 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45d6d1b60eda44139a2dae7de22c97b4'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:14.266235 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-18d3fdca-05b8-4902-8545-8c2d0a707242 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45d6d1b60eda44139a2dae7de22c97b4'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:14.266996 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-18d3fdca-05b8-4902-8545-8c2d0a707242 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45d6d1b60eda44139a2dae7de22c97b4'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:14.381326 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1108/2185] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:09:13 2022] DELETE /identity/v3/users/45d6d1b60eda44139a2dae7de22c97b4 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:09:14.386315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10b1b1-3eb6-4e9b-ab06-938f38ada9dd None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:14.387313 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10b1b1-3eb6-4e9b-ab06-938f38ada9dd None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:14.387809 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10b1b1-3eb6-4e9b-ab06-938f38ada9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:14.412345 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-1b10b1b1-3eb6-4e9b-ab06-938f38ada9dd None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:14.413733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-1b10b1b1-3eb6-4e9b-ab06-938f38ada9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:14.472626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:14.502724 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1078/2186] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:14.510360 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d3fdca-05b8-4902-8545-8c2d0a707242 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:14.511864 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-18d3fdca-05b8-4902-8545-8c2d0a707242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:14.623751 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d3fdca-05b8-4902-8545-8c2d0a707242 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:14.628914 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21945193-1d35-4aaa-ae04-9a910418b6d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:14.629971 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21945193-1d35-4aaa-ae04-9a910418b6d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:14.630476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21945193-1d35-4aaa-ae04-9a910418b6d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:14.630476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21945193-1d35-4aaa-ae04-9a910418b6d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:14.647191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21945193-1d35-4aaa-ae04-9a910418b6d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:14.648734 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21945193-1d35-4aaa-ae04-9a910418b6d4 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:14.659683 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1109/2187] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:09:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:09:15.246558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b10b1b1-3eb6-4e9b-ab06-938f38ada9dd None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:15.248777 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-1b10b1b1-3eb6-4e9b-ab06-938f38ada9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:15.268347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b10b1b1-3eb6-4e9b-ab06-938f38ada9dd None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:15.274794 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a8e3c-1658-496f-a233-df161637d9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:15.280500 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a8e3c-1658-496f-a233-df161637d9c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:15.280500 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a8e3c-1658-496f-a233-df161637d9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:15.280500 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a8e3c-1658-496f-a233-df161637d9c4 admin admin] PATH_INFO: `/v3/projects/9fad90825ade447992b855635af31169` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:15.280500 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a8e3c-1658-496f-a233-df161637d9c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fad90825ade447992b855635af31169)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:15.280500 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a8e3c-1658-496f-a233-df161637d9c4 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:15.425302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-1d6a8e3c-1658-496f-a233-df161637d9c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fad90825ade447992b855635af31169', 'request_id': 'req-1d6a8e3c-1658-496f-a233-df161637d9c4'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:15.434136 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-1d6a8e3c-1658-496f-a233-df161637d9c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fad90825ade447992b855635af31169', 'request_id': 'req-1d6a8e3c-1658-496f-a233-df161637d9c4'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:15.441573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-1d6a8e3c-1658-496f-a233-df161637d9c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fad90825ade447992b855635af31169', 'request_id': 'req-1d6a8e3c-1658-496f-a233-df161637d9c4'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:15.516970 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-1d6a8e3c-1658-496f-a233-df161637d9c4 admin admin] The token cache is being invalidate because project 9fad90825ade447992b855635af31169 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:09:15.517571 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-1d6a8e3c-1658-496f-a233-df161637d9c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:15.518750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-1d6a8e3c-1658-496f-a233-df161637d9c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:15.522194 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1079/2188] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:09:15 2022] DELETE /identity/v3/projects/9fad90825ade447992b855635af31169 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:09:15.527766 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21945193-1d35-4aaa-ae04-9a910418b6d4 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:15.541940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21945193-1d35-4aaa-ae04-9a910418b6d4 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:15.638156 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba568a-d1a0-4607-8ab7-442072021d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:15.638856 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba568a-d1a0-4607-8ab7-442072021d22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:15.639143 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba568a-d1a0-4607-8ab7-442072021d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:15.639464 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba568a-d1a0-4607-8ab7-442072021d22 admin admin] PATH_INFO: `/v3/users/7448de0fd25842f6909e8888f2a8b98c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:15.641365 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba568a-d1a0-4607-8ab7-442072021d22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7448de0fd25842f6909e8888f2a8b98c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:15.644815 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba568a-d1a0-4607-8ab7-442072021d22 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:16.066295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f2ba568a-d1a0-4607-8ab7-442072021d22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7448de0fd25842f6909e8888f2a8b98c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:16.076321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f2ba568a-d1a0-4607-8ab7-442072021d22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7448de0fd25842f6909e8888f2a8b98c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:16.077128 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f2ba568a-d1a0-4607-8ab7-442072021d22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7448de0fd25842f6909e8888f2a8b98c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:16.083982 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:16.103991 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f2ba568a-d1a0-4607-8ab7-442072021d22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7448de0fd25842f6909e8888f2a8b98c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:16.104870 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f2ba568a-d1a0-4607-8ab7-442072021d22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7448de0fd25842f6909e8888f2a8b98c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:16.242686 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1110/2189] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:09:15 2022] DELETE /identity/v3/users/7448de0fd25842f6909e8888f2a8b98c => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:09:16.353222 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a8e3c-1658-496f-a233-df161637d9c4 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:16.355630 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-1d6a8e3c-1658-496f-a233-df161637d9c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:16.373565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a8e3c-1658-496f-a233-df161637d9c4 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:16.444340 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fca6fa-2be8-4925-ab87-ab0564df55f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:16.445418 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fca6fa-2be8-4925-ab87-ab0564df55f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:16.445995 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fca6fa-2be8-4925-ab87-ab0564df55f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:16.446516 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fca6fa-2be8-4925-ab87-ab0564df55f0 admin admin] PATH_INFO: `/v3/users/964a1cb3b1f043088f074f8986183bb3` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:16.448493 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fca6fa-2be8-4925-ab87-ab0564df55f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=964a1cb3b1f043088f074f8986183bb3)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:16.452205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fca6fa-2be8-4925-ab87-ab0564df55f0 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:16.856753 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba568a-d1a0-4607-8ab7-442072021d22 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:16.865137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f2ba568a-d1a0-4607-8ab7-442072021d22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:16.871588 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-77fca6fa-2be8-4925-ab87-ab0564df55f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '964a1cb3b1f043088f074f8986183bb3'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:16.885178 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba568a-d1a0-4607-8ab7-442072021d22 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:16.888683 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-77fca6fa-2be8-4925-ab87-ab0564df55f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '964a1cb3b1f043088f074f8986183bb3'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:16.890361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-77fca6fa-2be8-4925-ab87-ab0564df55f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '964a1cb3b1f043088f074f8986183bb3'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:16.891264 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e52574-70b9-4217-ab75-4865d2ace11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:16.892229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e52574-70b9-4217-ab75-4865d2ace11d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:16.892706 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e52574-70b9-4217-ab75-4865d2ace11d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:16.893076 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e52574-70b9-4217-ab75-4865d2ace11d admin admin] PATH_INFO: `/v3/projects/febf5943abdd4a1ba0ee51f80c395e68` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:16.894670 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e52574-70b9-4217-ab75-4865d2ace11d admin admin] RBAC: Authorizing `identity:delete_project(project_id=febf5943abdd4a1ba0ee51f80c395e68)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:16.898382 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e52574-70b9-4217-ab75-4865d2ace11d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:17.171773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-77fca6fa-2be8-4925-ab87-ab0564df55f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '964a1cb3b1f043088f074f8986183bb3'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:17.190580 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:17.193176 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-77fca6fa-2be8-4925-ab87-ab0564df55f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '964a1cb3b1f043088f074f8986183bb3'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:17.201283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b0e52574-70b9-4217-ab75-4865d2ace11d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'febf5943abdd4a1ba0ee51f80c395e68', 'request_id': 'req-b0e52574-70b9-4217-ab75-4865d2ace11d'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:17.204264 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1080/2190] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:09:16 2022] DELETE /identity/v3/users/964a1cb3b1f043088f074f8986183bb3 => generated 0 bytes in 852 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:09:17.211282 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63d7fc-fc92-4787-83a5-f926411430fe None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:17.212898 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63d7fc-fc92-4787-83a5-f926411430fe None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:17.213832 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63d7fc-fc92-4787-83a5-f926411430fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:17.220698 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b0e52574-70b9-4217-ab75-4865d2ace11d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'febf5943abdd4a1ba0ee51f80c395e68', 'request_id': 'req-b0e52574-70b9-4217-ab75-4865d2ace11d'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:17.240859 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-ad63d7fc-fc92-4787-83a5-f926411430fe None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:17.242109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-ad63d7fc-fc92-4787-83a5-f926411430fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:17.312627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1081/2191] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:17.324057 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63d7fc-fc92-4787-83a5-f926411430fe None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:17.325699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-ad63d7fc-fc92-4787-83a5-f926411430fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:17.389514 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63d7fc-fc92-4787-83a5-f926411430fe None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:17.395833 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcbce02-f274-4652-a22a-a27d33d5cc22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:17.396812 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbce02-f274-4652-a22a-a27d33d5cc22 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:17.397289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbce02-f274-4652-a22a-a27d33d5cc22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:17.397739 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcbce02-f274-4652-a22a-a27d33d5cc22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:17.409867 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbce02-f274-4652-a22a-a27d33d5cc22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:17.412095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcbce02-f274-4652-a22a-a27d33d5cc22 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:17.468588 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1082/2192] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:09:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:09:17.514027 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b0e52574-70b9-4217-ab75-4865d2ace11d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'febf5943abdd4a1ba0ee51f80c395e68', 'request_id': 'req-b0e52574-70b9-4217-ab75-4865d2ace11d'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:17.533848 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b0e52574-70b9-4217-ab75-4865d2ace11d admin admin] The token cache is being invalidate because project febf5943abdd4a1ba0ee51f80c395e68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:09:17.534227 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b0e52574-70b9-4217-ab75-4865d2ace11d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:17.534995 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b0e52574-70b9-4217-ab75-4865d2ace11d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:17.536564 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1111/2193] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:09:16 2022] DELETE /identity/v3/projects/febf5943abdd4a1ba0ee51f80c395e68 => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:09:17.541870 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9c610-d4bc-4c2c-afbf-85dc3302eb84 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:17.542205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9c610-d4bc-4c2c-afbf-85dc3302eb84 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:17.542486 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9c610-d4bc-4c2c-afbf-85dc3302eb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:17.567287 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-18e9c610-d4bc-4c2c-afbf-85dc3302eb84 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:17.569120 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-18e9c610-d4bc-4c2c-afbf-85dc3302eb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:17.577395 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1083/2194] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:09:17.586106 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3ff4a-59c2-4433-8a16-e9ad58cc0929 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:17.586393 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3ff4a-59c2-4433-8a16-e9ad58cc0929 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:17.586677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3ff4a-59c2-4433-8a16-e9ad58cc0929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:17.607190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-0fd3ff4a-59c2-4433-8a16-e9ad58cc0929 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:17.608431 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0fd3ff4a-59c2-4433-8a16-e9ad58cc0929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:17.617490 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1112/2195] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:17.622091 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9c610-d4bc-4c2c-afbf-85dc3302eb84 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:17.632685 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9c610-d4bc-4c2c-afbf-85dc3302eb84 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:17.637629 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d083ea9-34ea-433e-91a6-fe56a4befdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:17.642574 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d083ea9-34ea-433e-91a6-fe56a4befdaf admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:17.642574 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d083ea9-34ea-433e-91a6-fe56a4befdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:17.642574 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d083ea9-34ea-433e-91a6-fe56a4befdaf admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:17.642574 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d083ea9-34ea-433e-91a6-fe56a4befdaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:09:17.642574 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d083ea9-34ea-433e-91a6-fe56a4befdaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:17.642574 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d083ea9-34ea-433e-91a6-fe56a4befdaf admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:17.653715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1084/2196] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:09:17.658331 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3ff4a-59c2-4433-8a16-e9ad58cc0929 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:17.668797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3ff4a-59c2-4433-8a16-e9ad58cc0929 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:17.673717 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05962cf0-1fb4-4516-8354-601981aaa761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:17.674448 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05962cf0-1fb4-4516-8354-601981aaa761 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:17.674733 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05962cf0-1fb4-4516-8354-601981aaa761 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:17.675012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05962cf0-1fb4-4516-8354-601981aaa761 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:17.675675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05962cf0-1fb4-4516-8354-601981aaa761 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:17.678148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05962cf0-1fb4-4516-8354-601981aaa761 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:17.899297 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1113/2197] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:09:17 2022] POST /identity/v3/projects => generated 385 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:09:17.912497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d083ea9-34ea-433e-91a6-fe56a4befdaf admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:17.930098 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d083ea9-34ea-433e-91a6-fe56a4befdaf admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:18.016012 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06aa84d-45d1-437c-9ca1-f8e42c9bd3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:18.021261 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06aa84d-45d1-437c-9ca1-f8e42c9bd3d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:18.021747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06aa84d-45d1-437c-9ca1-f8e42c9bd3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:18.022210 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06aa84d-45d1-437c-9ca1-f8e42c9bd3d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:18.023035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06aa84d-45d1-437c-9ca1-f8e42c9bd3d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:18.025128 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06aa84d-45d1-437c-9ca1-f8e42c9bd3d2 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:18.202330 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05962cf0-1fb4-4516-8354-601981aaa761 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:18.204366 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-05962cf0-1fb4-4516-8354-601981aaa761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:18.218062 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05962cf0-1fb4-4516-8354-601981aaa761 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:18.223744 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304bef3e-750a-4834-8b88-a4c82076a541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:18.225100 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bef3e-750a-4834-8b88-a4c82076a541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:18.225528 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bef3e-750a-4834-8b88-a4c82076a541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:18.225820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304bef3e-750a-4834-8b88-a4c82076a541 admin admin] PATH_INFO: `/v3/projects/2869c44d3e3d4d6aa69226f5ac3cf848` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:18.227054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bef3e-750a-4834-8b88-a4c82076a541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2869c44d3e3d4d6aa69226f5ac3cf848)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:18.228998 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304bef3e-750a-4834-8b88-a4c82076a541 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:18.264958 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1085/2198] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:09:17 2022] POST /identity/v3/users => generated 293 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:09:18.282114 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06aa84d-45d1-437c-9ca1-f8e42c9bd3d2 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:18.284055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06aa84d-45d1-437c-9ca1-f8e42c9bd3d2 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:18.290360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b8b4cc-6f75-4d0d-b551-196c4ff99c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:18.291182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b8b4cc-6f75-4d0d-b551-196c4ff99c6a admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:18.291467 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b8b4cc-6f75-4d0d-b551-196c4ff99c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:18.291751 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b8b4cc-6f75-4d0d-b551-196c4ff99c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:18.292463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b8b4cc-6f75-4d0d-b551-196c4ff99c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:18.294123 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b8b4cc-6f75-4d0d-b551-196c4ff99c6a admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:18.296791 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:18.309263 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1086/2199] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:09:18.313687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b8b4cc-6f75-4d0d-b551-196c4ff99c6a admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:18.325083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b8b4cc-6f75-4d0d-b551-196c4ff99c6a admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:18.330447 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1fcf7b-fc14-466c-8ecb-af19eff3a9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:18.331187 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fcf7b-fc14-466c-8ecb-af19eff3a9b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:18.331530 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fcf7b-fc14-466c-8ecb-af19eff3a9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:18.331851 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fcf7b-fc14-466c-8ecb-af19eff3a9b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:18.332652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1fcf7b-fc14-466c-8ecb-af19eff3a9b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:18.335361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1fcf7b-fc14-466c-8ecb-af19eff3a9b1 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:18.347241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1087/2200] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:09:18.354171 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1fcf7b-fc14-466c-8ecb-af19eff3a9b1 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:18.364051 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1fcf7b-fc14-466c-8ecb-af19eff3a9b1 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:18.370018 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0457dd-25bb-488e-a7c2-3932876d7926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:18.370677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0457dd-25bb-488e-a7c2-3932876d7926 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:18.370956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0457dd-25bb-488e-a7c2-3932876d7926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:18.371231 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0457dd-25bb-488e-a7c2-3932876d7926 admin admin] PATH_INFO: `/v3/projects/5831e1331e754754b07f93c22ca7b298/users/31dd4e4ae8ef40e2a4fde81872db2b1e/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:18.387924 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0457dd-25bb-488e-a7c2-3932876d7926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5831e1331e754754b07f93c22ca7b298, user_id=31dd4e4ae8ef40e2a4fde81872db2b1e, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:18.390772 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0457dd-25bb-488e-a7c2-3932876d7926 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:18.561034 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1088/2201] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:09:18 2022] PUT /identity/v3/projects/5831e1331e754754b07f93c22ca7b298/users/31dd4e4ae8ef40e2a4fde81872db2b1e/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:09:18.572095 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-304bef3e-750a-4834-8b88-a4c82076a541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2869c44d3e3d4d6aa69226f5ac3cf848', 'request_id': 'req-304bef3e-750a-4834-8b88-a4c82076a541'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:18.572711 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a8802-6e4a-4502-8c07-1aded75c2648 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:18.579323 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a8802-6e4a-4502-8c07-1aded75c2648 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:18.579323 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a8802-6e4a-4502-8c07-1aded75c2648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:18.579852 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-304bef3e-750a-4834-8b88-a4c82076a541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2869c44d3e3d4d6aa69226f5ac3cf848', 'request_id': 'req-304bef3e-750a-4834-8b88-a4c82076a541'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:18.608734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-ae9a8802-6e4a-4502-8c07-1aded75c2648 None None] MFA Rules not processed for user `31dd4e4ae8ef40e2a4fde81872db2b1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:18.610179 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-ae9a8802-6e4a-4502-8c07-1aded75c2648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:18.692152 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1089/2202] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:18.713685 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a8802-6e4a-4502-8c07-1aded75c2648 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:18.715900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-ae9a8802-6e4a-4502-8c07-1aded75c2648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:18.764644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-304bef3e-750a-4834-8b88-a4c82076a541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2869c44d3e3d4d6aa69226f5ac3cf848', 'request_id': 'req-304bef3e-750a-4834-8b88-a4c82076a541'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:18.788243 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-304bef3e-750a-4834-8b88-a4c82076a541 admin admin] The token cache is being invalidate because project 2869c44d3e3d4d6aa69226f5ac3cf848 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:09:18.788243 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-304bef3e-750a-4834-8b88-a4c82076a541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:18.788458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a8802-6e4a-4502-8c07-1aded75c2648 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:18.788549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-304bef3e-750a-4834-8b88-a4c82076a541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:18.794146 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1114/2203] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:09:18 2022] DELETE /identity/v3/projects/2869c44d3e3d4d6aa69226f5ac3cf848 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:09:18.797936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e0615-6723-4a75-a0f9-165a7bad8bc5 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:18.798251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e0615-6723-4a75-a0f9-165a7bad8bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:18.798535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e0615-6723-4a75-a0f9-165a7bad8bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:18.825168 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa191814-f43b-4c84-8c25-4f58bf18005a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:18.825827 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa191814-f43b-4c84-8c25-4f58bf18005a service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:18.826320 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa191814-f43b-4c84-8c25-4f58bf18005a service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:18.826743 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa191814-f43b-4c84-8c25-4f58bf18005a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:18.828996 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-fa191814-f43b-4c84-8c25-4f58bf18005a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:18.832401 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-de6e0615-6723-4a75-a0f9-165a7bad8bc5 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:18.835505 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-de6e0615-6723-4a75-a0f9-165a7bad8bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:18.862049 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa191814-f43b-4c84-8c25-4f58bf18005a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:18.863539 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa191814-f43b-4c84-8c25-4f58bf18005a service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:18.867246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:DELETE=17,keystone:SELECT=79,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:18.885958 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1090/2204] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:09:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:09:18.977322 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1115/2205] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:18.994362 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbcec6-797e-44da-ba29-1f2ac22459b1 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:18.994656 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbcec6-797e-44da-ba29-1f2ac22459b1 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:18.994938 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbcec6-797e-44da-ba29-1f2ac22459b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:19.018480 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-3acbcec6-797e-44da-ba29-1f2ac22459b1 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:19.019809 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-3acbcec6-797e-44da-ba29-1f2ac22459b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:19.030164 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1091/2206] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:19.035293 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e0615-6723-4a75-a0f9-165a7bad8bc5 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:19.056467 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e0615-6723-4a75-a0f9-165a7bad8bc5 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:19.062068 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1402d-7dba-4794-adf7-0679198cc6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:19.063903 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1402d-7dba-4794-adf7-0679198cc6eb admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:19.063903 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1402d-7dba-4794-adf7-0679198cc6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:19.063903 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b1402d-7dba-4794-adf7-0679198cc6eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:19.064650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b1402d-7dba-4794-adf7-0679198cc6eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:09:19.064999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b1402d-7dba-4794-adf7-0679198cc6eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:19.067228 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b1402d-7dba-4794-adf7-0679198cc6eb admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:19.095446 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1116/2207] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:09:19.096998 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbcec6-797e-44da-ba29-1f2ac22459b1 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:19.116352 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbcec6-797e-44da-ba29-1f2ac22459b1 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:19.123329 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92e6c1-bccc-4fac-9747-d53f49fc3702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:19.124033 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92e6c1-bccc-4fac-9747-d53f49fc3702 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:19.124330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92e6c1-bccc-4fac-9747-d53f49fc3702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:19.124834 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92e6c1-bccc-4fac-9747-d53f49fc3702 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:19.125718 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d92e6c1-bccc-4fac-9747-d53f49fc3702 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:19.128309 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d92e6c1-bccc-4fac-9747-d53f49fc3702 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:19.144807 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1402d-7dba-4794-adf7-0679198cc6eb admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:19.149918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-93b1402d-7dba-4794-adf7-0679198cc6eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:19.164248 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b1402d-7dba-4794-adf7-0679198cc6eb admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:19.170398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3db17-0dd4-4636-a193-ca039473dc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:19.171384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3db17-0dd4-4636-a193-ca039473dc9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:19.171942 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3db17-0dd4-4636-a193-ca039473dc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:19.172236 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3db17-0dd4-4636-a193-ca039473dc9d admin admin] PATH_INFO: `/v3/users/2c6f9cb461cc480dbc84496dfd3e2b5c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:19.174303 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd3db17-0dd4-4636-a193-ca039473dc9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c6f9cb461cc480dbc84496dfd3e2b5c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:19.180411 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd3db17-0dd4-4636-a193-ca039473dc9d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:19.279125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1092/2208] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:09:19 2022] POST /identity/v3/projects => generated 369 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:09:19.283453 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92e6c1-bccc-4fac-9747-d53f49fc3702 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:19.350546 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:19.393638 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92e6c1-bccc-4fac-9747-d53f49fc3702 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:19.399873 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a2742-855a-4170-b04c-541507f8f5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:19.400508 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a2742-855a-4170-b04c-541507f8f5f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:19.400824 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a2742-855a-4170-b04c-541507f8f5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:19.401253 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a2742-855a-4170-b04c-541507f8f5f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:19.401801 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a2742-855a-4170-b04c-541507f8f5f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:19.403781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a2742-855a-4170-b04c-541507f8f5f9 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:19.471455 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6bd3db17-0dd4-4636-a193-ca039473dc9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c6f9cb461cc480dbc84496dfd3e2b5c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:19.479496 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6bd3db17-0dd4-4636-a193-ca039473dc9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c6f9cb461cc480dbc84496dfd3e2b5c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:19.480250 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6bd3db17-0dd4-4636-a193-ca039473dc9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c6f9cb461cc480dbc84496dfd3e2b5c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:19.496475 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6bd3db17-0dd4-4636-a193-ca039473dc9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c6f9cb461cc480dbc84496dfd3e2b5c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:19.497753 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6bd3db17-0dd4-4636-a193-ca039473dc9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c6f9cb461cc480dbc84496dfd3e2b5c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:19.567059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1093/2209] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:09:19 2022] POST /identity/v3/users => generated 285 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:09:19.571929 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a2742-855a-4170-b04c-541507f8f5f9 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:19.579142 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-146a2742-855a-4170-b04c-541507f8f5f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:19.588792 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a2742-855a-4170-b04c-541507f8f5f9 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:19.593738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2054a93-2a09-49be-8869-c9ea883ac427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:19.594392 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2054a93-2a09-49be-8869-c9ea883ac427 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:19.594713 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2054a93-2a09-49be-8869-c9ea883ac427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:19.595022 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2054a93-2a09-49be-8869-c9ea883ac427 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:19.595771 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2054a93-2a09-49be-8869-c9ea883ac427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:19.597562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2054a93-2a09-49be-8869-c9ea883ac427 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:19.608603 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1094/2210] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:09:19.613196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2054a93-2a09-49be-8869-c9ea883ac427 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:19.623425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1117/2211] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:09:19 2022] DELETE /identity/v3/users/2c6f9cb461cc480dbc84496dfd3e2b5c => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:09:19.625216 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2054a93-2a09-49be-8869-c9ea883ac427 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:19.634258 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603e805-dead-4036-8182-aba9afd1ec35 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:19.634552 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603e805-dead-4036-8182-aba9afd1ec35 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:19.634901 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1603e805-dead-4036-8182-aba9afd1ec35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:19.658194 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-1603e805-dead-4036-8182-aba9afd1ec35 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:19.659642 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1603e805-dead-4036-8182-aba9afd1ec35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:19.688928 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f525f63-6c93-415c-9f7b-4167c68e4845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:19.689737 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f525f63-6c93-415c-9f7b-4167c68e4845 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:19.689737 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f525f63-6c93-415c-9f7b-4167c68e4845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:19.689943 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f525f63-6c93-415c-9f7b-4167c68e4845 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:19.690790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f525f63-6c93-415c-9f7b-4167c68e4845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:19.692831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f525f63-6c93-415c-9f7b-4167c68e4845 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:19.705503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1095/2212] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:09:19 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:09:19.710962 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f525f63-6c93-415c-9f7b-4167c68e4845 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:19.726698 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f525f63-6c93-415c-9f7b-4167c68e4845 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:19.732140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8287d-1bf3-4cbe-9e49-d086a37a404e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:19.732932 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8287d-1bf3-4cbe-9e49-d086a37a404e admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:19.733381 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8287d-1bf3-4cbe-9e49-d086a37a404e admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:19.733780 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1118/2213] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:09:19.739748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8287d-1bf3-4cbe-9e49-d086a37a404e admin admin] PATH_INFO: `/v3/projects/3096d9a95111415d893c6fc82ce860c4/users/d1381b81b5e14cfab800d6d3e8214f00/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:19.746360 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603e805-dead-4036-8182-aba9afd1ec35 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:19.748807 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1603e805-dead-4036-8182-aba9afd1ec35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:19.764164 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8287d-1bf3-4cbe-9e49-d086a37a404e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3096d9a95111415d893c6fc82ce860c4, user_id=d1381b81b5e14cfab800d6d3e8214f00, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:19.767480 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8287d-1bf3-4cbe-9e49-d086a37a404e admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:19.772005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:19.844183 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1603e805-dead-4036-8182-aba9afd1ec35 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:19.849085 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a4c4c4-4718-4cb4-ab48-6ce632380b17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:19.849855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4c4c4-4718-4cb4-ab48-6ce632380b17 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:19.850294 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4c4c4-4718-4cb4-ab48-6ce632380b17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:19.850689 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4c4c4-4718-4cb4-ab48-6ce632380b17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:19.871495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a4c4c4-4718-4cb4-ab48-6ce632380b17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:19.873731 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a4c4c4-4718-4cb4-ab48-6ce632380b17 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:19.890350 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1119/2214] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:09:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:09:19.945970 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40d252-d92f-409c-8ce1-199bffa4878b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:19.945970 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40d252-d92f-409c-8ce1-199bffa4878b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:19.945970 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40d252-d92f-409c-8ce1-199bffa4878b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:19.946375 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1096/2215] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:09:19 2022] PUT /identity/v3/projects/3096d9a95111415d893c6fc82ce860c4/users/d1381b81b5e14cfab800d6d3e8214f00/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:09:19.968129 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-1b40d252-d92f-409c-8ce1-199bffa4878b None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:19.973066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1b40d252-d92f-409c-8ce1-199bffa4878b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:20.079601 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1120/2216] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:20.094843 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8287d-1bf3-4cbe-9e49-d086a37a404e admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:20.175740 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8287d-1bf3-4cbe-9e49-d086a37a404e admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:20.187730 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160af309-285e-4033-b3d2-03fc06e5f079 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:20.188491 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160af309-285e-4033-b3d2-03fc06e5f079 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:20.188914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160af309-285e-4033-b3d2-03fc06e5f079 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:20.189218 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160af309-285e-4033-b3d2-03fc06e5f079 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:20.200538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160af309-285e-4033-b3d2-03fc06e5f079 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:20.202715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160af309-285e-4033-b3d2-03fc06e5f079 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:20.216623 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1097/2217] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:09:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:09:20.575098 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40d252-d92f-409c-8ce1-199bffa4878b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:20.578895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1b40d252-d92f-409c-8ce1-199bffa4878b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:20.595825 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40d252-d92f-409c-8ce1-199bffa4878b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:20.601767 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d09b2-94cc-4534-9218-ed7c1ce04a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:20.602758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d09b2-94cc-4534-9218-ed7c1ce04a3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:20.603168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d09b2-94cc-4534-9218-ed7c1ce04a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:20.605589 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d09b2-94cc-4534-9218-ed7c1ce04a3e admin admin] PATH_INFO: `/v3/projects/5763b774e5814ca38900cfb8bee110a0` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:20.606307 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6d09b2-94cc-4534-9218-ed7c1ce04a3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5763b774e5814ca38900cfb8bee110a0)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:20.609459 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6d09b2-94cc-4534-9218-ed7c1ce04a3e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:20.791838 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8e6d09b2-94cc-4534-9218-ed7c1ce04a3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5763b774e5814ca38900cfb8bee110a0', 'request_id': 'req-8e6d09b2-94cc-4534-9218-ed7c1ce04a3e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:20.805690 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8e6d09b2-94cc-4534-9218-ed7c1ce04a3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5763b774e5814ca38900cfb8bee110a0', 'request_id': 'req-8e6d09b2-94cc-4534-9218-ed7c1ce04a3e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:20.906410 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8e6d09b2-94cc-4534-9218-ed7c1ce04a3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5763b774e5814ca38900cfb8bee110a0', 'request_id': 'req-8e6d09b2-94cc-4534-9218-ed7c1ce04a3e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:20.921575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8e6d09b2-94cc-4534-9218-ed7c1ce04a3e admin admin] The token cache is being invalidate because project 5763b774e5814ca38900cfb8bee110a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:09:20.922351 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8e6d09b2-94cc-4534-9218-ed7c1ce04a3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:20.923192 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8e6d09b2-94cc-4534-9218-ed7c1ce04a3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:20.931949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1121/2218] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:09:20 2022] DELETE /identity/v3/projects/5763b774e5814ca38900cfb8bee110a0 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:09:20.936371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b4096-6420-4fe8-b1e8-8cb1ae0b9126 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:20.936371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b4096-6420-4fe8-b1e8-8cb1ae0b9126 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:20.936371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037b4096-6420-4fe8-b1e8-8cb1ae0b9126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:20.962222 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-037b4096-6420-4fe8-b1e8-8cb1ae0b9126 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:20.963447 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-037b4096-6420-4fe8-b1e8-8cb1ae0b9126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:20.981577 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:21.069626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1098/2219] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:21.148941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123490d-aef2-4d73-929c-67b5eeee54eb None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:21.150087 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123490d-aef2-4d73-929c-67b5eeee54eb None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:21.151463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e123490d-aef2-4d73-929c-67b5eeee54eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:21.189346 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-e123490d-aef2-4d73-929c-67b5eeee54eb None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:21.191384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e123490d-aef2-4d73-929c-67b5eeee54eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:21.200878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1122/2220] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:21.206194 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b4096-6420-4fe8-b1e8-8cb1ae0b9126 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:21.223156 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037b4096-6420-4fe8-b1e8-8cb1ae0b9126 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:21.234994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04e4af-4352-4148-9bb8-b73780269e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:21.235645 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04e4af-4352-4148-9bb8-b73780269e96 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:21.235929 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04e4af-4352-4148-9bb8-b73780269e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:21.236209 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04e4af-4352-4148-9bb8-b73780269e96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:21.236924 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04e4af-4352-4148-9bb8-b73780269e96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:09:21.237233 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04e4af-4352-4148-9bb8-b73780269e96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:21.238865 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04e4af-4352-4148-9bb8-b73780269e96 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:21.258254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1099/2221] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:09:21.261861 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123490d-aef2-4d73-929c-67b5eeee54eb None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:21.277736 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e123490d-aef2-4d73-929c-67b5eeee54eb None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:21.285580 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02f6cf-4465-4ea7-963c-a3174405e396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:21.287227 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02f6cf-4465-4ea7-963c-a3174405e396 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:21.287745 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02f6cf-4465-4ea7-963c-a3174405e396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:21.288082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b02f6cf-4465-4ea7-963c-a3174405e396 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:21.288950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02f6cf-4465-4ea7-963c-a3174405e396 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:21.291073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b02f6cf-4465-4ea7-963c-a3174405e396 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:21.492585 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1123/2222] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:09:21 2022] POST /identity/v3/projects => generated 375 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:09:21.497530 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04e4af-4352-4148-9bb8-b73780269e96 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:21.513861 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04e4af-4352-4148-9bb8-b73780269e96 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:21.609394 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4d471-c33c-4f92-89fe-75b718821930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:21.609817 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4d471-c33c-4f92-89fe-75b718821930 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:21.610134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4d471-c33c-4f92-89fe-75b718821930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:21.610412 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4d471-c33c-4f92-89fe-75b718821930 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:21.611093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4d471-c33c-4f92-89fe-75b718821930 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:21.613295 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4d471-c33c-4f92-89fe-75b718821930 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:21.635231 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02f6cf-4465-4ea7-963c-a3174405e396 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:21.637045 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0b02f6cf-4465-4ea7-963c-a3174405e396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:21.654573 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:21.711886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b02f6cf-4465-4ea7-963c-a3174405e396 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:21.715694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c38130-a4e1-419a-8d7e-308d1df06af3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:21.716327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c38130-a4e1-419a-8d7e-308d1df06af3 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:21.716608 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c38130-a4e1-419a-8d7e-308d1df06af3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:21.716889 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c38130-a4e1-419a-8d7e-308d1df06af3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:21.718695 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d7c38130-a4e1-419a-8d7e-308d1df06af3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:21.752396 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1100/2223] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:09:21 2022] POST /identity/v3/users => generated 288 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:09:21.753709 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4d471-c33c-4f92-89fe-75b718821930 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:21.769091 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4d471-c33c-4f92-89fe-75b718821930 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:21.776518 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841e3f5-5a58-4e70-b88a-c83564ec7355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:21.777558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841e3f5-5a58-4e70-b88a-c83564ec7355 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:21.778089 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841e3f5-5a58-4e70-b88a-c83564ec7355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:21.778565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841e3f5-5a58-4e70-b88a-c83564ec7355 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:21.779857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e841e3f5-5a58-4e70-b88a-c83564ec7355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:21.782417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e841e3f5-5a58-4e70-b88a-c83564ec7355 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:21.796539 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1101/2224] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:09:21.798969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c38130-a4e1-419a-8d7e-308d1df06af3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:21.800497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c38130-a4e1-419a-8d7e-308d1df06af3 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:21.803099 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841e3f5-5a58-4e70-b88a-c83564ec7355 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:21.812854 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1124/2225] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:09:21 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:09:21.825743 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841e3f5-5a58-4e70-b88a-c83564ec7355 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:21.825743 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d53de3-7385-4bd7-a4de-cafc41cad3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:21.826546 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d53de3-7385-4bd7-a4de-cafc41cad3b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:21.826929 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d53de3-7385-4bd7-a4de-cafc41cad3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:21.827563 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d53de3-7385-4bd7-a4de-cafc41cad3b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:21.828362 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d53de3-7385-4bd7-a4de-cafc41cad3b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:21.830246 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d53de3-7385-4bd7-a4de-cafc41cad3b6 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:21.841495 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1102/2226] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:09:21.846158 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c38130-a4e1-419a-8d7e-308d1df06af3 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:21.863409 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c38130-a4e1-419a-8d7e-308d1df06af3 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:21.877967 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b93319-2802-4e02-9a11-c32bde555185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:21.878670 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b93319-2802-4e02-9a11-c32bde555185 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:21.878993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b93319-2802-4e02-9a11-c32bde555185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:21.879301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b93319-2802-4e02-9a11-c32bde555185 admin admin] PATH_INFO: `/v3/projects/a622931a6e284ee1a512fcbfc61e1357/users/2e2b7380d83c4fcb857a54ae51b68ed0/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:21.902003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b93319-2802-4e02-9a11-c32bde555185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a622931a6e284ee1a512fcbfc61e1357, user_id=2e2b7380d83c4fcb857a54ae51b68ed0, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:21.905013 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b93319-2802-4e02-9a11-c32bde555185 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:22.031901 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1125/2227] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:09:21 2022] PUT /identity/v3/projects/a622931a6e284ee1a512fcbfc61e1357/users/2e2b7380d83c4fcb857a54ae51b68ed0/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:09:22.042540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf6f8a-62b5-4e06-87de-230870c2b0ad None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:22.043494 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf6f8a-62b5-4e06-87de-230870c2b0ad None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:22.043962 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf6f8a-62b5-4e06-87de-230870c2b0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:22.075017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-69bf6f8a-62b5-4e06-87de-230870c2b0ad None None] MFA Rules not processed for user `2e2b7380d83c4fcb857a54ae51b68ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:22.076251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-69bf6f8a-62b5-4e06-87de-230870c2b0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:22.193562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1103/2228] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:09:22.209850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b93319-2802-4e02-9a11-c32bde555185 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:22.216951 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e6b93319-2802-4e02-9a11-c32bde555185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:22.311694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b93319-2802-4e02-9a11-c32bde555185 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:22.321771 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1158b-bb4e-49ed-84f4-9d7da3624421 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:22.323064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1158b-bb4e-49ed-84f4-9d7da3624421 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:22.323401 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1158b-bb4e-49ed-84f4-9d7da3624421 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:22.323680 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1158b-bb4e-49ed-84f4-9d7da3624421 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:22.336803 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1158b-bb4e-49ed-84f4-9d7da3624421 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:22.338481 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1158b-bb4e-49ed-84f4-9d7da3624421 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:22.348220 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1126/2229] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:09:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:09:22.785014 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be8aa9-514d-4ff0-b59f-6ad88a6486fa None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:22.785535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be8aa9-514d-4ff0-b59f-6ad88a6486fa None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:22.786645 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be8aa9-514d-4ff0-b59f-6ad88a6486fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:22.796914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:22.834649 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-75be8aa9-514d-4ff0-b59f-6ad88a6486fa None None] MFA Rules not processed for user `d1381b81b5e14cfab800d6d3e8214f00`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:22.837194 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-75be8aa9-514d-4ff0-b59f-6ad88a6486fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:22.948005 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1104/2230] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:09:23.090675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1158b-bb4e-49ed-84f4-9d7da3624421 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:23.096156 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7ad1158b-bb4e-49ed-84f4-9d7da3624421 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:23.118083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1158b-bb4e-49ed-84f4-9d7da3624421 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:23.125143 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc11fc7-01d0-4b66-877e-5b22f1bae0db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:23.126221 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc11fc7-01d0-4b66-877e-5b22f1bae0db service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:23.126719 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc11fc7-01d0-4b66-877e-5b22f1bae0db service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:23.127211 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc11fc7-01d0-4b66-877e-5b22f1bae0db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:23.143408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc11fc7-01d0-4b66-877e-5b22f1bae0db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:23.146766 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc11fc7-01d0-4b66-877e-5b22f1bae0db service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:23.165203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1127/2231] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:09:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:09:31.063153 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ba96b-a7e2-449a-bb84-e533648b4ef4 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:31.063670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ba96b-a7e2-449a-bb84-e533648b4ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:31.063779 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ba96b-a7e2-449a-bb84-e533648b4ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:31.084251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-080ba96b-a7e2-449a-bb84-e533648b4ef4 None None] MFA Rules not processed for user `2e2b7380d83c4fcb857a54ae51b68ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:31.085922 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-080ba96b-a7e2-449a-bb84-e533648b4ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:31.094939 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1105/2232] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:31.371059 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989cf21-cb13-4e8a-81c9-2fccaf89bd22 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:31.372011 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989cf21-cb13-4e8a-81c9-2fccaf89bd22 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:31.372505 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f989cf21-cb13-4e8a-81c9-2fccaf89bd22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:31.397000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f989cf21-cb13-4e8a-81c9-2fccaf89bd22 None None] MFA Rules not processed for user `d1381b81b5e14cfab800d6d3e8214f00`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:31.398806 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f989cf21-cb13-4e8a-81c9-2fccaf89bd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:31.413082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1128/2233] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:33.681612 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ba96b-a7e2-449a-bb84-e533648b4ef4 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:33.683227 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-080ba96b-a7e2-449a-bb84-e533648b4ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:33.771997 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ba96b-a7e2-449a-bb84-e533648b4ef4 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:33.777676 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c5f051-1db2-42c6-aaae-715b90c849a7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad5cc9927374836b0cc74db85b686fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:33.778347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5f051-1db2-42c6-aaae-715b90c849a7 service placement] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:33.778634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5f051-1db2-42c6-aaae-715b90c849a7 service placement] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:33.778913 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5f051-1db2-42c6-aaae-715b90c849a7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:33.780755 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f4c5f051-1db2-42c6-aaae-715b90c849a7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:33.794989 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c5f051-1db2-42c6-aaae-715b90c849a7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:33.796579 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c5f051-1db2-42c6-aaae-715b90c849a7 service placement] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:33.807872 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1106/2234] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:09:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:09:34.762658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989cf21-cb13-4e8a-81c9-2fccaf89bd22 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:34.764083 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f989cf21-cb13-4e8a-81c9-2fccaf89bd22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:34.794288 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:34.840819 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f989cf21-cb13-4e8a-81c9-2fccaf89bd22 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:34.845550 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f2ead0-411a-4f13-8643-e74e9adad30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:34.846298 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2ead0-411a-4f13-8643-e74e9adad30f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:34.846573 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2ead0-411a-4f13-8643-e74e9adad30f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:34.846871 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f2ead0-411a-4f13-8643-e74e9adad30f admin admin] PATH_INFO: `/v3/users/31dd4e4ae8ef40e2a4fde81872db2b1e` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:34.848756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f2ead0-411a-4f13-8643-e74e9adad30f admin admin] RBAC: Authorizing `identity:delete_user(user_id=31dd4e4ae8ef40e2a4fde81872db2b1e)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:34.850845 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f2ead0-411a-4f13-8643-e74e9adad30f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:35.058352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-34f2ead0-411a-4f13-8643-e74e9adad30f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31dd4e4ae8ef40e2a4fde81872db2b1e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:35.065783 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-34f2ead0-411a-4f13-8643-e74e9adad30f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31dd4e4ae8ef40e2a4fde81872db2b1e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:35.066600 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-34f2ead0-411a-4f13-8643-e74e9adad30f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31dd4e4ae8ef40e2a4fde81872db2b1e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:35.080302 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-34f2ead0-411a-4f13-8643-e74e9adad30f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31dd4e4ae8ef40e2a4fde81872db2b1e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:35.081259 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-34f2ead0-411a-4f13-8643-e74e9adad30f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31dd4e4ae8ef40e2a4fde81872db2b1e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:35.199932 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1129/2235] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:09:34 2022] DELETE /identity/v3/users/31dd4e4ae8ef40e2a4fde81872db2b1e => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:09:35.204879 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342558b3-39dc-4eb1-862d-ffe3d2c53c42 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:35.205414 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342558b3-39dc-4eb1-862d-ffe3d2c53c42 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:35.205500 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342558b3-39dc-4eb1-862d-ffe3d2c53c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:35.243114 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-342558b3-39dc-4eb1-862d-ffe3d2c53c42 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:35.244644 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-342558b3-39dc-4eb1-862d-ffe3d2c53c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:35.325035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1107/2236] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:35.338361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f2ead0-411a-4f13-8643-e74e9adad30f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:35.339896 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-34f2ead0-411a-4f13-8643-e74e9adad30f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:35.405273 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f2ead0-411a-4f13-8643-e74e9adad30f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:35.412268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d5971-c7eb-4abe-be88-95629551fed5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:35.413072 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d5971-c7eb-4abe-be88-95629551fed5 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:35.413364 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d5971-c7eb-4abe-be88-95629551fed5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:35.413737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d5971-c7eb-4abe-be88-95629551fed5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:35.425169 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d5971-c7eb-4abe-be88-95629551fed5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:35.428411 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d5971-c7eb-4abe-be88-95629551fed5 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:35.447251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1130/2237] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:09:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:09:36.286605 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342558b3-39dc-4eb1-862d-ffe3d2c53c42 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:36.287502 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-342558b3-39dc-4eb1-862d-ffe3d2c53c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:36.302768 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342558b3-39dc-4eb1-862d-ffe3d2c53c42 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:36.309004 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07ad51-a74c-40b8-9aad-5ced0b4fa61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:36.309754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07ad51-a74c-40b8-9aad-5ced0b4fa61f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:36.310483 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07ad51-a74c-40b8-9aad-5ced0b4fa61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:36.310483 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07ad51-a74c-40b8-9aad-5ced0b4fa61f admin admin] PATH_INFO: `/v3/projects/5831e1331e754754b07f93c22ca7b298` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:36.311564 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07ad51-a74c-40b8-9aad-5ced0b4fa61f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5831e1331e754754b07f93c22ca7b298)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:36.313596 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07ad51-a74c-40b8-9aad-5ced0b4fa61f admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:36.338059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:36.494612 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-5d07ad51-a74c-40b8-9aad-5ced0b4fa61f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5831e1331e754754b07f93c22ca7b298', 'request_id': 'req-5d07ad51-a74c-40b8-9aad-5ced0b4fa61f'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:36.502522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-5d07ad51-a74c-40b8-9aad-5ced0b4fa61f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5831e1331e754754b07f93c22ca7b298', 'request_id': 'req-5d07ad51-a74c-40b8-9aad-5ced0b4fa61f'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:36.509927 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-5d07ad51-a74c-40b8-9aad-5ced0b4fa61f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5831e1331e754754b07f93c22ca7b298', 'request_id': 'req-5d07ad51-a74c-40b8-9aad-5ced0b4fa61f'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:36.586703 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-5d07ad51-a74c-40b8-9aad-5ced0b4fa61f admin admin] The token cache is being invalidate because project 5831e1331e754754b07f93c22ca7b298 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:09:36.587088 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-5d07ad51-a74c-40b8-9aad-5ced0b4fa61f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:36.587915 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-5d07ad51-a74c-40b8-9aad-5ced0b4fa61f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:36.589534 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1108/2238] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:09:36 2022] DELETE /identity/v3/projects/5831e1331e754754b07f93c22ca7b298 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:09:36.594437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db30a3b-5822-40d9-b20f-89aebf8fc3b7 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:36.595246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db30a3b-5822-40d9-b20f-89aebf8fc3b7 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:36.595680 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db30a3b-5822-40d9-b20f-89aebf8fc3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:36.617126 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-8db30a3b-5822-40d9-b20f-89aebf8fc3b7 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:36.618515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8db30a3b-5822-40d9-b20f-89aebf8fc3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:36.669518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:36.697618 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1131/2239] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:09:36.706141 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164833ae-9c4a-48fe-af04-c1c7d04d52ef None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:36.707210 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164833ae-9c4a-48fe-af04-c1c7d04d52ef None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:36.707317 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164833ae-9c4a-48fe-af04-c1c7d04d52ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:36.734458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-164833ae-9c4a-48fe-af04-c1c7d04d52ef None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:36.735810 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-164833ae-9c4a-48fe-af04-c1c7d04d52ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:36.746245 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1109/2240] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:36.755509 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db30a3b-5822-40d9-b20f-89aebf8fc3b7 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:36.766298 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db30a3b-5822-40d9-b20f-89aebf8fc3b7 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:36.772245 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a1648-9dc3-42c5-91b9-7e68297702e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:36.772977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a1648-9dc3-42c5-91b9-7e68297702e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:36.773330 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a1648-9dc3-42c5-91b9-7e68297702e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:36.773680 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a1648-9dc3-42c5-91b9-7e68297702e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:36.774639 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a1648-9dc3-42c5-91b9-7e68297702e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:09:36.775016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a1648-9dc3-42c5-91b9-7e68297702e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:36.776746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a1648-9dc3-42c5-91b9-7e68297702e6 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:36.791863 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1132/2241] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:09:36.796355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164833ae-9c4a-48fe-af04-c1c7d04d52ef None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:36.805957 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164833ae-9c4a-48fe-af04-c1c7d04d52ef None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:36.811433 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae46d8c4-bb05-45cc-a7c6-f9e535b0ad96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:36.812050 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46d8c4-bb05-45cc-a7c6-f9e535b0ad96 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:36.812510 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46d8c4-bb05-45cc-a7c6-f9e535b0ad96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:36.812953 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae46d8c4-bb05-45cc-a7c6-f9e535b0ad96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:36.813808 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae46d8c4-bb05-45cc-a7c6-f9e535b0ad96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:36.816286 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae46d8c4-bb05-45cc-a7c6-f9e535b0ad96 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:36.952208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1110/2242] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:09:36 2022] POST /identity/v3/projects => generated 377 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:09:36.957082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a1648-9dc3-42c5-91b9-7e68297702e6 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:36.968100 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a1648-9dc3-42c5-91b9-7e68297702e6 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:37.037058 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d16f1-ccc3-4c3d-ba87-50dad7a21a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:37.038122 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d16f1-ccc3-4c3d-ba87-50dad7a21a6f admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:37.038563 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d16f1-ccc3-4c3d-ba87-50dad7a21a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:37.039022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d16f1-ccc3-4c3d-ba87-50dad7a21a6f admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:37.039866 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d16f1-ccc3-4c3d-ba87-50dad7a21a6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:37.042006 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d16f1-ccc3-4c3d-ba87-50dad7a21a6f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:37.156479 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1133/2243] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:09:36 2022] POST /identity/v3/users => generated 289 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:09:37.157915 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae46d8c4-bb05-45cc-a7c6-f9e535b0ad96 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:37.171414 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae46d8c4-bb05-45cc-a7c6-f9e535b0ad96 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:37.179098 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb1b65-a679-4a52-a38d-6156eb9b6294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:37.180499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb1b65-a679-4a52-a38d-6156eb9b6294 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:37.180956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb1b65-a679-4a52-a38d-6156eb9b6294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:37.181334 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb1b65-a679-4a52-a38d-6156eb9b6294 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:37.182156 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb1b65-a679-4a52-a38d-6156eb9b6294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:37.184205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fb1b65-a679-4a52-a38d-6156eb9b6294 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:37.202678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1111/2244] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:09:37.204044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d16f1-ccc3-4c3d-ba87-50dad7a21a6f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:37.215309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d16f1-ccc3-4c3d-ba87-50dad7a21a6f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:37.220181 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8092050a-50fa-4476-af66-97a6e14f40e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:37.220808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092050a-50fa-4476-af66-97a6e14f40e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:37.221093 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092050a-50fa-4476-af66-97a6e14f40e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:37.221370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8092050a-50fa-4476-af66-97a6e14f40e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:37.222250 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092050a-50fa-4476-af66-97a6e14f40e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:37.223843 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8092050a-50fa-4476-af66-97a6e14f40e6 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:37.236954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1134/2245] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:09:37.241779 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb1b65-a679-4a52-a38d-6156eb9b6294 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:37.252697 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fb1b65-a679-4a52-a38d-6156eb9b6294 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:37.258432 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fec9c4-d17c-4fc6-8b07-870316debfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:37.259270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fec9c4-d17c-4fc6-8b07-870316debfea admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:37.259559 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fec9c4-d17c-4fc6-8b07-870316debfea admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:37.259837 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fec9c4-d17c-4fc6-8b07-870316debfea admin admin] PATH_INFO: `/v3/projects/9e1368624f6e46fd970127425fb7b4dd/users/e93dd81c1ff6475d9a6df9d801783f20/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:37.283767 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed185338-ba34-429f-8dbb-65f9046fe394 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:37.284088 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed185338-ba34-429f-8dbb-65f9046fe394 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:37.284478 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed185338-ba34-429f-8dbb-65f9046fe394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:37.286532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fec9c4-d17c-4fc6-8b07-870316debfea admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e1368624f6e46fd970127425fb7b4dd, user_id=e93dd81c1ff6475d9a6df9d801783f20, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:37.289858 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fec9c4-d17c-4fc6-8b07-870316debfea admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:37.307542 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-ed185338-ba34-429f-8dbb-65f9046fe394 None None] MFA Rules not processed for user `7fd6b148658e4c2884a32f66326833b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:37.308945 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ed185338-ba34-429f-8dbb-65f9046fe394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:37.380114 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1112/2246] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:09:37 2022] PUT /identity/v3/projects/9e1368624f6e46fd970127425fb7b4dd/users/e93dd81c1ff6475d9a6df9d801783f20/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:09:37.386180 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1135/2247] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:37 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:09:37.390790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57354ed8-1235-4700-84d6-6738a20b17e7 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:37.391193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57354ed8-1235-4700-84d6-6738a20b17e7 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:37.391562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57354ed8-1235-4700-84d6-6738a20b17e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:37.424924 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-57354ed8-1235-4700-84d6-6738a20b17e7 None None] MFA Rules not processed for user `e93dd81c1ff6475d9a6df9d801783f20`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:37.426863 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-57354ed8-1235-4700-84d6-6738a20b17e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:37.525449 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1113/2248] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:09:37.530110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed185338-ba34-429f-8dbb-65f9046fe394 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:37.542000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed185338-ba34-429f-8dbb-65f9046fe394 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:37.636959 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1648c156-5553-48d9-9f76-634004b7b7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:37.637855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648c156-5553-48d9-9f76-634004b7b7a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:37.638709 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648c156-5553-48d9-9f76-634004b7b7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:37.639725 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1648c156-5553-48d9-9f76-634004b7b7a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:37.640619 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1648c156-5553-48d9-9f76-634004b7b7a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:37.642971 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1648c156-5553-48d9-9f76-634004b7b7a9 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:37.678552 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:37.768378 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1136/2249] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:09:37 2022] POST /identity/v3/projects => generated 377 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:09:37.770414 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57354ed8-1235-4700-84d6-6738a20b17e7 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:37.782197 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57354ed8-1235-4700-84d6-6738a20b17e7 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:37.813574 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:37.855458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f78fe-50fc-47f2-963b-8fc4961a6ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:37.856208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f78fe-50fc-47f2-963b-8fc4961a6ab7 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:37.856553 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f78fe-50fc-47f2-963b-8fc4961a6ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:37.856885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f78fe-50fc-47f2-963b-8fc4961a6ab7 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:37.859583 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f78fe-50fc-47f2-963b-8fc4961a6ab7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:37.860222 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f78fe-50fc-47f2-963b-8fc4961a6ab7 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:37.948364 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1114/2250] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:09:37 2022] POST /identity/v3/users => generated 289 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:09:37.953479 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1648c156-5553-48d9-9f76-634004b7b7a9 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:37.965314 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1648c156-5553-48d9-9f76-634004b7b7a9 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:37.970277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec884c-52c0-42f0-9a34-1124cdad9114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:37.971022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec884c-52c0-42f0-9a34-1124cdad9114 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:37.971404 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec884c-52c0-42f0-9a34-1124cdad9114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:37.971778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ec884c-52c0-42f0-9a34-1124cdad9114 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:37.973125 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec884c-52c0-42f0-9a34-1124cdad9114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:37.974923 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ec884c-52c0-42f0-9a34-1124cdad9114 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:37.994778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1137/2251] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:09:37.995365 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f78fe-50fc-47f2-963b-8fc4961a6ab7 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:38.007983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f78fe-50fc-47f2-963b-8fc4961a6ab7 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:38.013688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7aed5-63c7-48aa-8e60-64cd8a11f683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:38.014356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7aed5-63c7-48aa-8e60-64cd8a11f683 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:38.015015 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7aed5-63c7-48aa-8e60-64cd8a11f683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:38.015015 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7aed5-63c7-48aa-8e60-64cd8a11f683 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:38.015596 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7aed5-63c7-48aa-8e60-64cd8a11f683 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:38.017269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7aed5-63c7-48aa-8e60-64cd8a11f683 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:38.031402 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1115/2252] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:09:38.034091 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec884c-52c0-42f0-9a34-1124cdad9114 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:38.046003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ec884c-52c0-42f0-9a34-1124cdad9114 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:38.056115 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf5835-945f-4d9a-a2a9-71feca69fcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:38.056320 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf5835-945f-4d9a-a2a9-71feca69fcdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:38.056694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf5835-945f-4d9a-a2a9-71feca69fcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:38.057067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf5835-945f-4d9a-a2a9-71feca69fcdb admin admin] PATH_INFO: `/v3/projects/1f96993042b345b6b9e84a9f95e16123/users/e746c25c4fee4d2aabd814c4d4fd5140/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:38.077512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf5835-945f-4d9a-a2a9-71feca69fcdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f96993042b345b6b9e84a9f95e16123, user_id=e746c25c4fee4d2aabd814c4d4fd5140, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:38.080255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaf5835-945f-4d9a-a2a9-71feca69fcdb admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:38.174502 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1138/2253] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:09:38 2022] PUT /identity/v3/projects/1f96993042b345b6b9e84a9f95e16123/users/e746c25c4fee4d2aabd814c4d4fd5140/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:09:38.186875 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990dd5b-e99b-4dbb-8782-f9c187450d93 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:38.187004 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990dd5b-e99b-4dbb-8782-f9c187450d93 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:38.187242 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9990dd5b-e99b-4dbb-8782-f9c187450d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:38.229181 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-9990dd5b-e99b-4dbb-8782-f9c187450d93 None None] MFA Rules not processed for user `e746c25c4fee4d2aabd814c4d4fd5140`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:38.230974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-9990dd5b-e99b-4dbb-8782-f9c187450d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:38.318044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1116/2254] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:38 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:38.322179 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf5835-945f-4d9a-a2a9-71feca69fcdb admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:38.419574 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf5835-945f-4d9a-a2a9-71feca69fcdb admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:38.424785 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1bc4b-f5f3-4083-b88b-d8d8c053f6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:38.425528 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1bc4b-f5f3-4083-b88b-d8d8c053f6d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:38.425812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1bc4b-f5f3-4083-b88b-d8d8c053f6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:38.426120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1bc4b-f5f3-4083-b88b-d8d8c053f6d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:38.427047 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1bc4b-f5f3-4083-b88b-d8d8c053f6d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:38.429045 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1bc4b-f5f3-4083-b88b-d8d8c053f6d1 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:38.627931 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1139/2255] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:09:38 2022] POST /identity/v3/projects => generated 377 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:09:38.629273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990dd5b-e99b-4dbb-8782-f9c187450d93 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:38.641441 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9990dd5b-e99b-4dbb-8782-f9c187450d93 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:38.722859 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a90af-ca9f-4b56-94dc-1b2a21177b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:38.723462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a90af-ca9f-4b56-94dc-1b2a21177b5d admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:38.723746 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a90af-ca9f-4b56-94dc-1b2a21177b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:38.724023 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554a90af-ca9f-4b56-94dc-1b2a21177b5d admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:38.725815 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a90af-ca9f-4b56-94dc-1b2a21177b5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:38.727450 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554a90af-ca9f-4b56-94dc-1b2a21177b5d admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:38.836456 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1bc4b-f5f3-4083-b88b-d8d8c053f6d1 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:38.849085 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1117/2256] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:09:38 2022] POST /identity/v3/users => generated 295 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:09:38.864856 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1bc4b-f5f3-4083-b88b-d8d8c053f6d1 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:38.873808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391144d-da7a-45c9-87ef-78d441ffafef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:38.874977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391144d-da7a-45c9-87ef-78d441ffafef admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:38.875423 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391144d-da7a-45c9-87ef-78d441ffafef admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:38.875860 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391144d-da7a-45c9-87ef-78d441ffafef admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:38.876746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e391144d-da7a-45c9-87ef-78d441ffafef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:38.879108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e391144d-da7a-45c9-87ef-78d441ffafef admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:38.909167 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a90af-ca9f-4b56-94dc-1b2a21177b5d admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:38.910170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1140/2257] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:09:38 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:09:38.921448 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554a90af-ca9f-4b56-94dc-1b2a21177b5d admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:38.934386 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7dac8a-c7ac-4350-bef9-6b700ce5abc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:38.935180 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7dac8a-c7ac-4350-bef9-6b700ce5abc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:38.935574 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7dac8a-c7ac-4350-bef9-6b700ce5abc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:38.935927 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7dac8a-c7ac-4350-bef9-6b700ce5abc5 admin admin] PATH_INFO: `/v3/projects/0d6a8d394aa649038dd8c3755118d0db/users/24205cce5ed5498295075654908f158a/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:38.959662 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7dac8a-c7ac-4350-bef9-6b700ce5abc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d6a8d394aa649038dd8c3755118d0db, user_id=24205cce5ed5498295075654908f158a, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:38.963935 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7dac8a-c7ac-4350-bef9-6b700ce5abc5 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:39.063135 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:39.069476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1118/2258] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:09:38 2022] PUT /identity/v3/projects/0d6a8d394aa649038dd8c3755118d0db/users/24205cce5ed5498295075654908f158a/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:09:39.073250 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391144d-da7a-45c9-87ef-78d441ffafef admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:39.094536 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391144d-da7a-45c9-87ef-78d441ffafef admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:39.210718 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38e867-edff-46fe-ba0a-79fcf18c267d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:39.211509 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38e867-edff-46fe-ba0a-79fcf18c267d admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:39.211889 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38e867-edff-46fe-ba0a-79fcf18c267d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:39.212265 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38e867-edff-46fe-ba0a-79fcf18c267d admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:39.213213 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38e867-edff-46fe-ba0a-79fcf18c267d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:39.215514 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38e867-edff-46fe-ba0a-79fcf18c267d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:39.233663 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1141/2259] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:09:39.236808 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7dac8a-c7ac-4350-bef9-6b700ce5abc5 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:39.262947 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7dac8a-c7ac-4350-bef9-6b700ce5abc5 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:39.269695 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1cfab-793a-4703-affd-0908bfcc6430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:39.270850 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1cfab-793a-4703-affd-0908bfcc6430 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:39.271314 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1cfab-793a-4703-affd-0908bfcc6430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:39.272574 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1cfab-793a-4703-affd-0908bfcc6430 admin admin] PATH_INFO: `/v3/projects/0d6a8d394aa649038dd8c3755118d0db/users/24205cce5ed5498295075654908f158a/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:39.274430 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1cfab-793a-4703-affd-0908bfcc6430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d6a8d394aa649038dd8c3755118d0db, user_id=24205cce5ed5498295075654908f158a, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:39.277717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1cfab-793a-4703-affd-0908bfcc6430 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:39.289511 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1119/2260] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:09:39 2022] PUT /identity/v3/projects/0d6a8d394aa649038dd8c3755118d0db/users/24205cce5ed5498295075654908f158a/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:09:39.301260 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09006cd3-628b-44fb-96c9-ec825a188214 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:39.301663 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09006cd3-628b-44fb-96c9-ec825a188214 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:39.302126 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09006cd3-628b-44fb-96c9-ec825a188214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:39.335403 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-09006cd3-628b-44fb-96c9-ec825a188214 None None] MFA Rules not processed for user `24205cce5ed5498295075654908f158a`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:39.336657 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-09006cd3-628b-44fb-96c9-ec825a188214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:39.346378 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:39.421496 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1142/2261] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:39.433290 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1cfab-793a-4703-affd-0908bfcc6430 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:39.435118 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4da1cfab-793a-4703-affd-0908bfcc6430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:39.494757 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1cfab-793a-4703-affd-0908bfcc6430 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:39.498699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc957774-7c83-411e-8f4d-633f780555c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:39.505040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc957774-7c83-411e-8f4d-633f780555c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:39.505040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc957774-7c83-411e-8f4d-633f780555c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:39.505040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc957774-7c83-411e-8f4d-633f780555c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:39.559478 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc957774-7c83-411e-8f4d-633f780555c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:39.561087 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc957774-7c83-411e-8f4d-633f780555c9 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:39.570987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1120/2262] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:09:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:09:42.783990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09006cd3-628b-44fb-96c9-ec825a188214 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:42.794746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09006cd3-628b-44fb-96c9-ec825a188214 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:42.799658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91211143-6a38-44f8-840b-96e3a3b7229c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:42.801046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91211143-6a38-44f8-840b-96e3a3b7229c service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:42.801360 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91211143-6a38-44f8-840b-96e3a3b7229c service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:42.801723 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91211143-6a38-44f8-840b-96e3a3b7229c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:42.892629 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91211143-6a38-44f8-840b-96e3a3b7229c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:42.896170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91211143-6a38-44f8-840b-96e3a3b7229c service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:42.913713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1143/2263] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:09:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:09:44.751180 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc957774-7c83-411e-8f4d-633f780555c9 service cinder] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:44.761207 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc957774-7c83-411e-8f4d-633f780555c9 service cinder] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:44.765052 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b31777-829b-4c38-ac31-0ea77ff42887 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:44.765795 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b31777-829b-4c38-ac31-0ea77ff42887 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:44.766090 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b31777-829b-4c38-ac31-0ea77ff42887 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:44.766368 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b31777-829b-4c38-ac31-0ea77ff42887 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:44.779064 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b31777-829b-4c38-ac31-0ea77ff42887 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:44.780522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b31777-829b-4c38-ac31-0ea77ff42887 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:44.792311 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1121/2264] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:09:44 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:09:51.133974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91211143-6a38-44f8-840b-96e3a3b7229c service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:51.265797 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91211143-6a38-44f8-840b-96e3a3b7229c service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:51.272641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b507ce-243d-4bb2-ad3d-a3f87c748d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:51.273925 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b507ce-243d-4bb2-ad3d-a3f87c748d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:51.274386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b507ce-243d-4bb2-ad3d-a3f87c748d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:51.274819 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b507ce-243d-4bb2-ad3d-a3f87c748d6b admin admin] PATH_INFO: `/v3/users/e93dd81c1ff6475d9a6df9d801783f20` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:51.276931 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b507ce-243d-4bb2-ad3d-a3f87c748d6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e93dd81c1ff6475d9a6df9d801783f20)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:51.281274 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b507ce-243d-4bb2-ad3d-a3f87c748d6b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:51.571852 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:51.578717 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-32b507ce-243d-4bb2-ad3d-a3f87c748d6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e93dd81c1ff6475d9a6df9d801783f20'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:51.589132 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-32b507ce-243d-4bb2-ad3d-a3f87c748d6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e93dd81c1ff6475d9a6df9d801783f20'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:51.590099 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-32b507ce-243d-4bb2-ad3d-a3f87c748d6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e93dd81c1ff6475d9a6df9d801783f20'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:51.607367 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-32b507ce-243d-4bb2-ad3d-a3f87c748d6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e93dd81c1ff6475d9a6df9d801783f20'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:51.608381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-32b507ce-243d-4bb2-ad3d-a3f87c748d6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e93dd81c1ff6475d9a6df9d801783f20'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:51.717484 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1144/2265] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:09:51 2022] DELETE /identity/v3/users/e93dd81c1ff6475d9a6df9d801783f20 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:09:51.722643 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff42a46-3405-4fb0-8aca-bcf6597d92da None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:51.723303 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff42a46-3405-4fb0-8aca-bcf6597d92da None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:51.723303 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff42a46-3405-4fb0-8aca-bcf6597d92da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:51.744065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-cff42a46-3405-4fb0-8aca-bcf6597d92da None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:51.745275 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-cff42a46-3405-4fb0-8aca-bcf6597d92da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:51.816544 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1122/2266] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:51.823803 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b507ce-243d-4bb2-ad3d-a3f87c748d6b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:51.825274 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-32b507ce-243d-4bb2-ad3d-a3f87c748d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:51.894398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b507ce-243d-4bb2-ad3d-a3f87c748d6b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:51.898469 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbfd946-d513-4031-b54d-235434ac8ba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:51.899075 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbfd946-d513-4031-b54d-235434ac8ba3 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:51.899389 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbfd946-d513-4031-b54d-235434ac8ba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:51.899657 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbfd946-d513-4031-b54d-235434ac8ba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:51.910188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbfd946-d513-4031-b54d-235434ac8ba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:51.911698 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbfd946-d513-4031-b54d-235434ac8ba3 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:51.925631 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1145/2267] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:09:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:09:52.372288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff42a46-3405-4fb0-8aca-bcf6597d92da None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:52.373755 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-cff42a46-3405-4fb0-8aca-bcf6597d92da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:52.388436 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff42a46-3405-4fb0-8aca-bcf6597d92da None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:52.393393 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc4e17-8728-4170-8bb4-37d725eb0b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:52.394087 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc4e17-8728-4170-8bb4-37d725eb0b0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:52.394370 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc4e17-8728-4170-8bb4-37d725eb0b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:52.394647 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc4e17-8728-4170-8bb4-37d725eb0b0e admin admin] PATH_INFO: `/v3/projects/9e1368624f6e46fd970127425fb7b4dd` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:52.396780 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc4e17-8728-4170-8bb4-37d725eb0b0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e1368624f6e46fd970127425fb7b4dd)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:52.399185 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc4e17-8728-4170-8bb4-37d725eb0b0e admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:52.430700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:52.527382 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9ddc4e17-8728-4170-8bb4-37d725eb0b0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e1368624f6e46fd970127425fb7b4dd', 'request_id': 'req-9ddc4e17-8728-4170-8bb4-37d725eb0b0e'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:52.537109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9ddc4e17-8728-4170-8bb4-37d725eb0b0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e1368624f6e46fd970127425fb7b4dd', 'request_id': 'req-9ddc4e17-8728-4170-8bb4-37d725eb0b0e'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:52.550759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9ddc4e17-8728-4170-8bb4-37d725eb0b0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e1368624f6e46fd970127425fb7b4dd', 'request_id': 'req-9ddc4e17-8728-4170-8bb4-37d725eb0b0e'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:52.665304 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9ddc4e17-8728-4170-8bb4-37d725eb0b0e admin admin] The token cache is being invalidate because project 9e1368624f6e46fd970127425fb7b4dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:09:52.666552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9ddc4e17-8728-4170-8bb4-37d725eb0b0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:52.667772 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9ddc4e17-8728-4170-8bb4-37d725eb0b0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:52.671595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1123/2268] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:09:52 2022] DELETE /identity/v3/projects/9e1368624f6e46fd970127425fb7b4dd => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:09:52.677051 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbfd946-d513-4031-b54d-235434ac8ba3 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:52.698887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbfd946-d513-4031-b54d-235434ac8ba3 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:52.802522 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8e94f-704a-4a97-97b0-42cdff8ee935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:52.803462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8e94f-704a-4a97-97b0-42cdff8ee935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:52.803902 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8e94f-704a-4a97-97b0-42cdff8ee935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:52.804401 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8e94f-704a-4a97-97b0-42cdff8ee935 admin admin] PATH_INFO: `/v3/users/e746c25c4fee4d2aabd814c4d4fd5140` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:52.806156 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae8e94f-704a-4a97-97b0-42cdff8ee935 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e746c25c4fee4d2aabd814c4d4fd5140)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:52.811128 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae8e94f-704a-4a97-97b0-42cdff8ee935 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:53.017202 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fae8e94f-704a-4a97-97b0-42cdff8ee935 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e746c25c4fee4d2aabd814c4d4fd5140'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:53.024768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fae8e94f-704a-4a97-97b0-42cdff8ee935 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e746c25c4fee4d2aabd814c4d4fd5140'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:53.025651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fae8e94f-704a-4a97-97b0-42cdff8ee935 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e746c25c4fee4d2aabd814c4d4fd5140'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:53.031632 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:53.039282 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fae8e94f-704a-4a97-97b0-42cdff8ee935 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e746c25c4fee4d2aabd814c4d4fd5140'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:53.040021 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fae8e94f-704a-4a97-97b0-42cdff8ee935 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e746c25c4fee4d2aabd814c4d4fd5140'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:53.117189 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1146/2269] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:09:52 2022] DELETE /identity/v3/users/e746c25c4fee4d2aabd814c4d4fd5140 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:09:53.650476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc4e17-8728-4170-8bb4-37d725eb0b0e admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:53.651709 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-9ddc4e17-8728-4170-8bb4-37d725eb0b0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:53.671823 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc4e17-8728-4170-8bb4-37d725eb0b0e admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:53.756108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f3342f-4eb4-401d-b618-4651ede81367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:53.756771 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3342f-4eb4-401d-b618-4651ede81367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:53.757063 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3342f-4eb4-401d-b618-4651ede81367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:53.757346 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3342f-4eb4-401d-b618-4651ede81367 admin admin] PATH_INFO: `/v3/projects/1f96993042b345b6b9e84a9f95e16123` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:53.758577 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f3342f-4eb4-401d-b618-4651ede81367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f96993042b345b6b9e84a9f95e16123)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:53.760520 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f3342f-4eb4-401d-b618-4651ede81367 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:53.915607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-18f3342f-4eb4-401d-b618-4651ede81367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f96993042b345b6b9e84a9f95e16123', 'request_id': 'req-18f3342f-4eb4-401d-b618-4651ede81367'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:53.929659 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-18f3342f-4eb4-401d-b618-4651ede81367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f96993042b345b6b9e84a9f95e16123', 'request_id': 'req-18f3342f-4eb4-401d-b618-4651ede81367'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:53.938493 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-18f3342f-4eb4-401d-b618-4651ede81367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f96993042b345b6b9e84a9f95e16123', 'request_id': 'req-18f3342f-4eb4-401d-b618-4651ede81367'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:54.031293 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-18f3342f-4eb4-401d-b618-4651ede81367 admin admin] The token cache is being invalidate because project 1f96993042b345b6b9e84a9f95e16123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:09:54.031627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-18f3342f-4eb4-401d-b618-4651ede81367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:54.032312 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-18f3342f-4eb4-401d-b618-4651ede81367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:54.034026 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1124/2270] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:09:53 2022] DELETE /identity/v3/projects/1f96993042b345b6b9e84a9f95e16123 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:09:54.038144 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8e94f-704a-4a97-97b0-42cdff8ee935 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:54.039618 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-fae8e94f-704a-4a97-97b0-42cdff8ee935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:54.053251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8e94f-704a-4a97-97b0-42cdff8ee935 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:54.118855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c3db2-850f-468f-bf96-be49131d70f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:54.119549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c3db2-850f-468f-bf96-be49131d70f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:54.119827 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c3db2-850f-468f-bf96-be49131d70f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:54.120107 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c3db2-850f-468f-bf96-be49131d70f5 admin admin] PATH_INFO: `/v3/users/24205cce5ed5498295075654908f158a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:54.121321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c3db2-850f-468f-bf96-be49131d70f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24205cce5ed5498295075654908f158a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:54.123363 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c3db2-850f-468f-bf96-be49131d70f5 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:54.358611 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5e7c3db2-850f-468f-bf96-be49131d70f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24205cce5ed5498295075654908f158a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:54.366157 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5e7c3db2-850f-468f-bf96-be49131d70f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24205cce5ed5498295075654908f158a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:54.366992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5e7c3db2-850f-468f-bf96-be49131d70f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24205cce5ed5498295075654908f158a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:54.385341 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5e7c3db2-850f-468f-bf96-be49131d70f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24205cce5ed5498295075654908f158a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:54.388494 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5e7c3db2-850f-468f-bf96-be49131d70f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24205cce5ed5498295075654908f158a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:54.492629 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1147/2271] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:09:54 2022] DELETE /identity/v3/users/24205cce5ed5498295075654908f158a => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:09:54.974050 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f3342f-4eb4-401d-b618-4651ede81367 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:54.975556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-18f3342f-4eb4-401d-b618-4651ede81367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:54.991093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f3342f-4eb4-401d-b618-4651ede81367 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:55.038052 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:55.057601 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5feffce-f0e3-4dcb-8a29-11824de1b7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:55.058385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5feffce-f0e3-4dcb-8a29-11824de1b7d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:55.058677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5feffce-f0e3-4dcb-8a29-11824de1b7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:55.058956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5feffce-f0e3-4dcb-8a29-11824de1b7d6 admin admin] PATH_INFO: `/v3/projects/0d6a8d394aa649038dd8c3755118d0db` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:55.060176 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5feffce-f0e3-4dcb-8a29-11824de1b7d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d6a8d394aa649038dd8c3755118d0db)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:55.062228 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5feffce-f0e3-4dcb-8a29-11824de1b7d6 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:55.196960 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-c5feffce-f0e3-4dcb-8a29-11824de1b7d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d6a8d394aa649038dd8c3755118d0db', 'request_id': 'req-c5feffce-f0e3-4dcb-8a29-11824de1b7d6'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:55.204095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-c5feffce-f0e3-4dcb-8a29-11824de1b7d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d6a8d394aa649038dd8c3755118d0db', 'request_id': 'req-c5feffce-f0e3-4dcb-8a29-11824de1b7d6'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:55.217232 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-c5feffce-f0e3-4dcb-8a29-11824de1b7d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d6a8d394aa649038dd8c3755118d0db', 'request_id': 'req-c5feffce-f0e3-4dcb-8a29-11824de1b7d6'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:55.288899 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-c5feffce-f0e3-4dcb-8a29-11824de1b7d6 admin admin] The token cache is being invalidate because project 0d6a8d394aa649038dd8c3755118d0db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:09:55.289181 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-c5feffce-f0e3-4dcb-8a29-11824de1b7d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:55.289961 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-c5feffce-f0e3-4dcb-8a29-11824de1b7d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:09:55.291926 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1125/2272] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:09:54 2022] DELETE /identity/v3/projects/0d6a8d394aa649038dd8c3755118d0db => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:09:55.297220 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486eaeb1-fd20-4ab5-b2ae-18b322945c68 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:55.297748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486eaeb1-fd20-4ab5-b2ae-18b322945c68 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:55.297932 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486eaeb1-fd20-4ab5-b2ae-18b322945c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:55.319138 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-486eaeb1-fd20-4ab5-b2ae-18b322945c68 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:55.320381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-486eaeb1-fd20-4ab5-b2ae-18b322945c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:55.375558 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:09:55.384581 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1148/2273] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:55.392321 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df84844-5e45-4b89-9e19-689b8f4dadf1 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:55.392616 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df84844-5e45-4b89-9e19-689b8f4dadf1 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:55.393746 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df84844-5e45-4b89-9e19-689b8f4dadf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:55.416710 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-3df84844-5e45-4b89-9e19-689b8f4dadf1 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:55.418005 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-3df84844-5e45-4b89-9e19-689b8f4dadf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:55.426704 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1126/2274] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:55.431229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486eaeb1-fd20-4ab5-b2ae-18b322945c68 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:55.442725 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486eaeb1-fd20-4ab5-b2ae-18b322945c68 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:55.447469 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381cf347-9458-4233-bc1b-88bb139351bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:55.448022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cf347-9458-4233-bc1b-88bb139351bd admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:55.448300 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cf347-9458-4233-bc1b-88bb139351bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:55.448569 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cf347-9458-4233-bc1b-88bb139351bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:55.449264 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381cf347-9458-4233-bc1b-88bb139351bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:09:55.449564 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381cf347-9458-4233-bc1b-88bb139351bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:55.451545 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381cf347-9458-4233-bc1b-88bb139351bd admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:55.464528 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1149/2275] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:09:55.470138 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df84844-5e45-4b89-9e19-689b8f4dadf1 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:55.479298 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df84844-5e45-4b89-9e19-689b8f4dadf1 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:55.485496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937702e-21c2-4139-bee9-c5abe4596d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:55.486060 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937702e-21c2-4139-bee9-c5abe4596d29 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:55.486389 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937702e-21c2-4139-bee9-c5abe4596d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:55.486758 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4937702e-21c2-4139-bee9-c5abe4596d29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:55.487451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937702e-21c2-4139-bee9-c5abe4596d29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:55.489434 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4937702e-21c2-4139-bee9-c5abe4596d29 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:55.609621 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1127/2276] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:09:55 2022] POST /identity/v3/projects => generated 365 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:09:55.611148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381cf347-9458-4233-bc1b-88bb139351bd admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:55.625954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381cf347-9458-4233-bc1b-88bb139351bd admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:55.724643 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c8e2d-ba81-4b52-8978-92a699f1aa3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:55.725644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c8e2d-ba81-4b52-8978-92a699f1aa3c admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:55.726194 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c8e2d-ba81-4b52-8978-92a699f1aa3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:55.726709 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c8e2d-ba81-4b52-8978-92a699f1aa3c admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:55.728102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c8e2d-ba81-4b52-8978-92a699f1aa3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:55.731294 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c8e2d-ba81-4b52-8978-92a699f1aa3c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:55.871982 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1150/2277] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:09:55 2022] POST /identity/v3/users => generated 283 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:09:55.879041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937702e-21c2-4139-bee9-c5abe4596d29 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:55.888705 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4937702e-21c2-4139-bee9-c5abe4596d29 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:55.894216 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81689f31-8ae5-47c1-9eb0-4182844a6112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:55.894836 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81689f31-8ae5-47c1-9eb0-4182844a6112 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:55.895118 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81689f31-8ae5-47c1-9eb0-4182844a6112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:55.895414 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81689f31-8ae5-47c1-9eb0-4182844a6112 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:55.896216 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81689f31-8ae5-47c1-9eb0-4182844a6112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:55.897726 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81689f31-8ae5-47c1-9eb0-4182844a6112 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:55.908654 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1128/2278] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:09:55.912845 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c8e2d-ba81-4b52-8978-92a699f1aa3c admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:55.929860 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c8e2d-ba81-4b52-8978-92a699f1aa3c admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:55.939761 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82c377-cced-4a83-b7ac-ed882219037c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:55.942098 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82c377-cced-4a83-b7ac-ed882219037c admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:55.942098 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82c377-cced-4a83-b7ac-ed882219037c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:55.942098 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82c377-cced-4a83-b7ac-ed882219037c admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:55.942891 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82c377-cced-4a83-b7ac-ed882219037c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:55.943818 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82c377-cced-4a83-b7ac-ed882219037c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:55.955773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1151/2279] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:09:55.960176 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81689f31-8ae5-47c1-9eb0-4182844a6112 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:55.970796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81689f31-8ae5-47c1-9eb0-4182844a6112 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:55.976227 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec77d2-53d5-45e5-8ac3-25ac13fb744d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:55.976988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec77d2-53d5-45e5-8ac3-25ac13fb744d admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:55.977353 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec77d2-53d5-45e5-8ac3-25ac13fb744d admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:55.977732 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bec77d2-53d5-45e5-8ac3-25ac13fb744d admin admin] PATH_INFO: `/v3/projects/828d7ba5e0394b3da1bb3ad9b6ca37c6/users/db84b8a008ef4f46964c336a0ce87c43/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:55.994786 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec77d2-53d5-45e5-8ac3-25ac13fb744d admin admin] RBAC: Authorizing `identity:create_grant(project_id=828d7ba5e0394b3da1bb3ad9b6ca37c6, user_id=db84b8a008ef4f46964c336a0ce87c43, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:55.997464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bec77d2-53d5-45e5-8ac3-25ac13fb744d admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:56.089454 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1129/2280] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:09:55 2022] PUT /identity/v3/projects/828d7ba5e0394b3da1bb3ad9b6ca37c6/users/db84b8a008ef4f46964c336a0ce87c43/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:09:56.162961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0245a-64c9-4e29-892a-fb609554defb None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:56.163247 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0245a-64c9-4e29-892a-fb609554defb None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:56.163529 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0245a-64c9-4e29-892a-fb609554defb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:56.194508 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-95f0245a-64c9-4e29-892a-fb609554defb None None] MFA Rules not processed for user `db84b8a008ef4f46964c336a0ce87c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:09:56.195779 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-95f0245a-64c9-4e29-892a-fb609554defb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:56.281506 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1152/2281] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:09:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:09:56.295510 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec77d2-53d5-45e5-8ac3-25ac13fb744d admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:09:56.297296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-3bec77d2-53d5-45e5-8ac3-25ac13fb744d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:09:56.392037 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bec77d2-53d5-45e5-8ac3-25ac13fb744d admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:09:56.396447 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b25c29-3124-4901-9c5a-3b208828390e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:09:56.397684 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b25c29-3124-4901-9c5a-3b208828390e service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:09:56.398148 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b25c29-3124-4901-9c5a-3b208828390e service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:09:56.398661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b25c29-3124-4901-9c5a-3b208828390e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:09:56.408210 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b25c29-3124-4901-9c5a-3b208828390e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:09:56.410201 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b25c29-3124-4901-9c5a-3b208828390e service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:09:56.430082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1130/2282] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:09:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:10:06.277836 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=72,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:06.415885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:09.740606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee87673-52ff-49ae-9c4d-763c2d2ab31d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:09.744555 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee87673-52ff-49ae-9c4d-763c2d2ab31d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:09.744555 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee87673-52ff-49ae-9c4d-763c2d2ab31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:09.762115 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-bee87673-52ff-49ae-9c4d-763c2d2ab31d None None] MFA Rules not processed for user `d1381b81b5e14cfab800d6d3e8214f00`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:09.767382 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bee87673-52ff-49ae-9c4d-763c2d2ab31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:09.822957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1153/2283] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:10:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:09.950391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8100eef-7901-4ada-88e0-0439cdc550af None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:09.950391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8100eef-7901-4ada-88e0-0439cdc550af None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:09.950391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8100eef-7901-4ada-88e0-0439cdc550af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:09.971496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-a8100eef-7901-4ada-88e0-0439cdc550af None None] MFA Rules not processed for user `ed8114b7a28741268fdff71c866dbc77`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:09.972708 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-a8100eef-7901-4ada-88e0-0439cdc550af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:10.029989 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1131/2284] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 2 09:10:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:10.094035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee87673-52ff-49ae-9c4d-763c2d2ab31d None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:10.096104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bee87673-52ff-49ae-9c4d-763c2d2ab31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:10.175180 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee87673-52ff-49ae-9c4d-763c2d2ab31d None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:10.182659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5d960-a719-4785-9523-bb92119a4a2a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff92ef3b9da74ccab3fbc43f852e3ad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:10.184268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5d960-a719-4785-9523-bb92119a4a2a service swift] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:10.184654 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5d960-a719-4785-9523-bb92119a4a2a service swift] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:10.185122 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c5d960-a719-4785-9523-bb92119a4a2a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:10.194484 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c5d960-a719-4785-9523-bb92119a4a2a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:10.196839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c5d960-a719-4785-9523-bb92119a4a2a service swift] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:10.208883 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1154/2285] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 2 09:10:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:10:11.207111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c7313-26ff-4bc4-a8ba-15913d65dff6 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:11.207613 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c7313-26ff-4bc4-a8ba-15913d65dff6 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:11.207901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5c7313-26ff-4bc4-a8ba-15913d65dff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:11.229994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-aa5c7313-26ff-4bc4-a8ba-15913d65dff6 None None] MFA Rules not processed for user `ed8114b7a28741268fdff71c866dbc77`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:11.231216 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-aa5c7313-26ff-4bc4-a8ba-15913d65dff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:11.343399 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1132/2286] 10.222.0.46 () {58 vars in 967 bytes} [Tue Aug 2 09:10:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:11.346775 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5d960-a719-4785-9523-bb92119a4a2a service swift] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:11.346775 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c5d960-a719-4785-9523-bb92119a4a2a service swift] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:11.346775 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be6a9c-b4aa-47ef-849d-390b19612afa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff92ef3b9da74ccab3fbc43f852e3ad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:11.346775 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be6a9c-b4aa-47ef-849d-390b19612afa service swift] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:11.348250 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be6a9c-b4aa-47ef-849d-390b19612afa service swift] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:11.348250 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35be6a9c-b4aa-47ef-849d-390b19612afa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:11.348250 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be6a9c-b4aa-47ef-849d-390b19612afa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:11.348250 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35be6a9c-b4aa-47ef-849d-390b19612afa service swift] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:11.348250 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1155/2287] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 2 09:10:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:10:12.131869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ee4a8-f6a9-4499-9ae4-8fbcb61801f5 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:12.132781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ee4a8-f6a9-4499-9ae4-8fbcb61801f5 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:12.132781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ee4a8-f6a9-4499-9ae4-8fbcb61801f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:12.153114 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-df2ee4a8-f6a9-4499-9ae4-8fbcb61801f5 None None] MFA Rules not processed for user `ed8114b7a28741268fdff71c866dbc77`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:12.154387 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-df2ee4a8-f6a9-4499-9ae4-8fbcb61801f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:12.161675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1133/2288] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 2 09:10:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:12.173921 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be6a9c-b4aa-47ef-849d-390b19612afa service swift] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:12.186679 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35be6a9c-b4aa-47ef-849d-390b19612afa service swift] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:12.194058 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdab218-d4ab-4a44-8fe6-14da1c4ecf86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff92ef3b9da74ccab3fbc43f852e3ad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:12.195092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdab218-d4ab-4a44-8fe6-14da1c4ecf86 service swift] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:12.195579 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdab218-d4ab-4a44-8fe6-14da1c4ecf86 service swift] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:12.196052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdab218-d4ab-4a44-8fe6-14da1c4ecf86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:12.213307 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdab218-d4ab-4a44-8fe6-14da1c4ecf86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:12.213307 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdab218-d4ab-4a44-8fe6-14da1c4ecf86 service swift] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:12.221640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1156/2289] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 2 09:10:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:10:12.851845 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c9e42-f3fa-42f3-a095-fe68519c5b49 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:12.852322 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c9e42-f3fa-42f3-a095-fe68519c5b49 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:12.852424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c9e42-f3fa-42f3-a095-fe68519c5b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:12.872166 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-3c5c9e42-f3fa-42f3-a095-fe68519c5b49 None None] MFA Rules not processed for user `ed8114b7a28741268fdff71c866dbc77`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:12.873359 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-3c5c9e42-f3fa-42f3-a095-fe68519c5b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:12.881398 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1134/2290] 10.222.0.46 () {58 vars in 967 bytes} [Tue Aug 2 09:10:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:12.893398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdab218-d4ab-4a44-8fe6-14da1c4ecf86 service swift] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:12.904176 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdab218-d4ab-4a44-8fe6-14da1c4ecf86 service swift] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:12.909830 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ba321f-5b8f-4130-8718-a85dc8b14121 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff92ef3b9da74ccab3fbc43f852e3ad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:12.910503 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba321f-5b8f-4130-8718-a85dc8b14121 service swift] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:12.910783 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba321f-5b8f-4130-8718-a85dc8b14121 service swift] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:12.911062 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ba321f-5b8f-4130-8718-a85dc8b14121 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:12.919379 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba321f-5b8f-4130-8718-a85dc8b14121 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:12.920978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ba321f-5b8f-4130-8718-a85dc8b14121 service swift] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:12.931080 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1157/2291] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 2 09:10:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:10:22.867418 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=34 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:22.926593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=75 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:24.226558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c9e42-f3fa-42f3-a095-fe68519c5b49 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:24.228440 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-3c5c9e42-f3fa-42f3-a095-fe68519c5b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:24.303643 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c9e42-f3fa-42f3-a095-fe68519c5b49 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:24.309065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a0c9d-b6d4-4140-899b-ee1064160e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:24.309725 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a0c9d-b6d4-4140-899b-ee1064160e61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:24.310036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a0c9d-b6d4-4140-899b-ee1064160e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:24.310322 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a0c9d-b6d4-4140-899b-ee1064160e61 admin admin] PATH_INFO: `/v3/users/2e2b7380d83c4fcb857a54ae51b68ed0` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:24.311523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a0c9d-b6d4-4140-899b-ee1064160e61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e2b7380d83c4fcb857a54ae51b68ed0)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:24.314335 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a0c9d-b6d4-4140-899b-ee1064160e61 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:24.612201 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-4e5a0c9d-b6d4-4140-899b-ee1064160e61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e2b7380d83c4fcb857a54ae51b68ed0'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:24.620444 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-4e5a0c9d-b6d4-4140-899b-ee1064160e61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e2b7380d83c4fcb857a54ae51b68ed0'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:24.621169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-4e5a0c9d-b6d4-4140-899b-ee1064160e61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e2b7380d83c4fcb857a54ae51b68ed0'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:24.692116 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-4e5a0c9d-b6d4-4140-899b-ee1064160e61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e2b7380d83c4fcb857a54ae51b68ed0'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:24.713428 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-4e5a0c9d-b6d4-4140-899b-ee1064160e61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e2b7380d83c4fcb857a54ae51b68ed0'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:24.717273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1135/2292] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:10:24 2022] DELETE /identity/v3/users/2e2b7380d83c4fcb857a54ae51b68ed0 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:10:24.719427 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c679a0e-4e24-47e3-87c3-1a7c76ee74bf None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:24.719894 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c679a0e-4e24-47e3-87c3-1a7c76ee74bf None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:24.720016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c679a0e-4e24-47e3-87c3-1a7c76ee74bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:24.740538 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5c679a0e-4e24-47e3-87c3-1a7c76ee74bf None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:24.742010 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5c679a0e-4e24-47e3-87c3-1a7c76ee74bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:24.815446 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1158/2293] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:10:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:24.825588 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a0c9d-b6d4-4140-899b-ee1064160e61 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:24.827754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4e5a0c9d-b6d4-4140-899b-ee1064160e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:24.895080 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a0c9d-b6d4-4140-899b-ee1064160e61 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:24.899365 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19745dcd-b847-4333-8926-c5332c5e51e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:24.900000 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19745dcd-b847-4333-8926-c5332c5e51e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:24.900278 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19745dcd-b847-4333-8926-c5332c5e51e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:24.900557 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19745dcd-b847-4333-8926-c5332c5e51e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:24.913690 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19745dcd-b847-4333-8926-c5332c5e51e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:24.915195 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19745dcd-b847-4333-8926-c5332c5e51e0 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:24.925722 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1136/2294] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:10:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:10:25.182584 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c679a0e-4e24-47e3-87c3-1a7c76ee74bf None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:25.184066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5c679a0e-4e24-47e3-87c3-1a7c76ee74bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:25.196930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c679a0e-4e24-47e3-87c3-1a7c76ee74bf None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:25.202244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37962e78-556b-4b66-a61f-841a2a9e52a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:25.203669 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37962e78-556b-4b66-a61f-841a2a9e52a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:25.203976 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37962e78-556b-4b66-a61f-841a2a9e52a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:25.204261 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37962e78-556b-4b66-a61f-841a2a9e52a3 admin admin] PATH_INFO: `/v3/projects/a622931a6e284ee1a512fcbfc61e1357` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:25.206302 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37962e78-556b-4b66-a61f-841a2a9e52a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a622931a6e284ee1a512fcbfc61e1357)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:25.208349 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37962e78-556b-4b66-a61f-841a2a9e52a3 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:25.344165 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-37962e78-556b-4b66-a61f-841a2a9e52a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a622931a6e284ee1a512fcbfc61e1357', 'request_id': 'req-37962e78-556b-4b66-a61f-841a2a9e52a3'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:25.350692 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-37962e78-556b-4b66-a61f-841a2a9e52a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a622931a6e284ee1a512fcbfc61e1357', 'request_id': 'req-37962e78-556b-4b66-a61f-841a2a9e52a3'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:25.435406 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-37962e78-556b-4b66-a61f-841a2a9e52a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a622931a6e284ee1a512fcbfc61e1357', 'request_id': 'req-37962e78-556b-4b66-a61f-841a2a9e52a3'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:25.449495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-37962e78-556b-4b66-a61f-841a2a9e52a3 admin admin] The token cache is being invalidate because project a622931a6e284ee1a512fcbfc61e1357 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:10:25.449820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-37962e78-556b-4b66-a61f-841a2a9e52a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:25.450509 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-37962e78-556b-4b66-a61f-841a2a9e52a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:10:25.452110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1159/2295] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:10:25 2022] DELETE /identity/v3/projects/a622931a6e284ee1a512fcbfc61e1357 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:10:34.920886 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:35.445720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:44.899173 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96dd85-83b2-49dc-9ebc-eff4b9b7dddc None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:44.899173 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96dd85-83b2-49dc-9ebc-eff4b9b7dddc None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:44.899173 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96dd85-83b2-49dc-9ebc-eff4b9b7dddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:44.919736 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-9e96dd85-83b2-49dc-9ebc-eff4b9b7dddc None None] MFA Rules not processed for user `806f0c03435e4c5daeadaa4eb437d936`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:44.920960 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-9e96dd85-83b2-49dc-9ebc-eff4b9b7dddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:44.994715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1137/2296] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:10:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:54.987793 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=25 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:10:55.211721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bd39f-3ea9-4171-a0ab-926e42a37cbe None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:55.212326 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bd39f-3ea9-4171-a0ab-926e42a37cbe None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:55.212326 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165bd39f-3ea9-4171-a0ab-926e42a37cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:55.232413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-165bd39f-3ea9-4171-a0ab-926e42a37cbe None None] MFA Rules not processed for user `ed8114b7a28741268fdff71c866dbc77`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:10:55.233639 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-165bd39f-3ea9-4171-a0ab-926e42a37cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:55.300713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1160/2297] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 2 09:10:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:10:55.314189 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96dd85-83b2-49dc-9ebc-eff4b9b7dddc None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:10:55.315713 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-9e96dd85-83b2-49dc-9ebc-eff4b9b7dddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:10:55.521441 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96dd85-83b2-49dc-9ebc-eff4b9b7dddc None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:10:55.526627 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdb715-6a41-40ee-8c0b-aa5ddb23fc66 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff92ef3b9da74ccab3fbc43f852e3ad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:10:55.527246 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdb715-6a41-40ee-8c0b-aa5ddb23fc66 service swift] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:10:55.527528 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdb715-6a41-40ee-8c0b-aa5ddb23fc66 service swift] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:10:55.527805 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdb715-6a41-40ee-8c0b-aa5ddb23fc66 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:10:55.536651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fdb715-6a41-40ee-8c0b-aa5ddb23fc66 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:10:55.538270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fdb715-6a41-40ee-8c0b-aa5ddb23fc66 service swift] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:10:55.546527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1138/2298] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 2 09:10:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:11:05.293748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=25 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:05.543178 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=32 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:16.336873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf9827-dc23-4afa-b981-b0f73d8cc9d8 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:16.337355 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf9827-dc23-4afa-b981-b0f73d8cc9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:16.337449 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cf9827-dc23-4afa-b981-b0f73d8cc9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:16.357435 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-79cf9827-dc23-4afa-b981-b0f73d8cc9d8 None None] MFA Rules not processed for user `ed8114b7a28741268fdff71c866dbc77`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:16.358657 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-79cf9827-dc23-4afa-b981-b0f73d8cc9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:16.365840 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1161/2299] 10.222.0.46 () {58 vars in 967 bytes} [Tue Aug 2 09:11:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:16.378025 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdb715-6a41-40ee-8c0b-aa5ddb23fc66 service swift] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:16.388314 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdb715-6a41-40ee-8c0b-aa5ddb23fc66 service swift] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:16.393532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e8132-2a2a-4528-bd67-436d6050e37e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff92ef3b9da74ccab3fbc43f852e3ad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:16.394159 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e8132-2a2a-4528-bd67-436d6050e37e service swift] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:16.394439 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e8132-2a2a-4528-bd67-436d6050e37e service swift] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:16.398897 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352e8132-2a2a-4528-bd67-436d6050e37e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:16.404084 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e8132-2a2a-4528-bd67-436d6050e37e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:16.405638 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352e8132-2a2a-4528-bd67-436d6050e37e service swift] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:16.414191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1139/2300] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 2 09:11:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:11:16.451023 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59eecc-c483-44b7-9a7d-b343768699cc None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:16.451408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59eecc-c483-44b7-9a7d-b343768699cc None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:16.451592 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59eecc-c483-44b7-9a7d-b343768699cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:16.470832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-7d59eecc-c483-44b7-9a7d-b343768699cc None None] MFA Rules not processed for user `f1fd384cbf4943368ee57e868c37d92c`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:16.472025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7d59eecc-c483-44b7-9a7d-b343768699cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:16.528159 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1162/2301] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:11:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:16.547261 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e8132-2a2a-4528-bd67-436d6050e37e service swift] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:16.548648 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-352e8132-2a2a-4528-bd67-436d6050e37e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:16.569873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59eecc-c483-44b7-9a7d-b343768699cc None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:16.571718 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7d59eecc-c483-44b7-9a7d-b343768699cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:16.612124 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352e8132-2a2a-4528-bd67-436d6050e37e service swift] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:16.615738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4400e90e-8f87-4056-b574-35e7f8cc158a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:16.616348 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400e90e-8f87-4056-b574-35e7f8cc158a service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:16.616624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400e90e-8f87-4056-b574-35e7f8cc158a service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:16.616892 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4400e90e-8f87-4056-b574-35e7f8cc158a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:16.618600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4400e90e-8f87-4056-b574-35e7f8cc158a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:16.635438 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59eecc-c483-44b7-9a7d-b343768699cc None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:16.639783 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b3f69-edee-4329-af54-532ab630f2b7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4abd1512ad224da896c6f2257f9602e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:16.640404 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b3f69-edee-4329-af54-532ab630f2b7 service glance] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:16.640800 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b3f69-edee-4329-af54-532ab630f2b7 service glance] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:16.640953 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521b3f69-edee-4329-af54-532ab630f2b7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:16.642745 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-521b3f69-edee-4329-af54-532ab630f2b7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:16.690649 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4400e90e-8f87-4056-b574-35e7f8cc158a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:16.692625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4400e90e-8f87-4056-b574-35e7f8cc158a service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:16.704063 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1140/2302] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:11:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:16.739871 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b3f69-edee-4329-af54-532ab630f2b7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:16.741327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521b3f69-edee-4329-af54-532ab630f2b7 service glance] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:16.761795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1163/2303] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:11:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:11:19.019019 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54535fc1-f923-4f65-a346-7886333fef54 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:19.019561 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54535fc1-f923-4f65-a346-7886333fef54 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:19.019649 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54535fc1-f923-4f65-a346-7886333fef54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:19.039859 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-54535fc1-f923-4f65-a346-7886333fef54 None None] MFA Rules not processed for user `ed8114b7a28741268fdff71c866dbc77`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:19.041165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-54535fc1-f923-4f65-a346-7886333fef54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:19.054843 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1141/2304] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 2 09:11:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:19.067300 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b3f69-edee-4329-af54-532ab630f2b7 service glance] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:19.080355 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521b3f69-edee-4329-af54-532ab630f2b7 service glance] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:19.086556 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4bf4e-b374-4563-aa03-91857c9f4b02 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff92ef3b9da74ccab3fbc43f852e3ad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:19.087294 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4bf4e-b374-4563-aa03-91857c9f4b02 service swift] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:19.087684 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4bf4e-b374-4563-aa03-91857c9f4b02 service swift] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:19.088062 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4bf4e-b374-4563-aa03-91857c9f4b02 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:19.097491 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4bf4e-b374-4563-aa03-91857c9f4b02 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:19.099794 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4bf4e-b374-4563-aa03-91857c9f4b02 service swift] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:19.111299 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1164/2305] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 2 09:11:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:11:19.702246 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54535fc1-f923-4f65-a346-7886333fef54 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:19.703880 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-54535fc1-f923-4f65-a346-7886333fef54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:19.817070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54535fc1-f923-4f65-a346-7886333fef54 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:19.843394 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38262d13-f7b8-4cf1-8d42-fe092ef73b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:19.844219 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38262d13-f7b8-4cf1-8d42-fe092ef73b10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:19.844501 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38262d13-f7b8-4cf1-8d42-fe092ef73b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:19.845182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38262d13-f7b8-4cf1-8d42-fe092ef73b10 admin admin] PATH_INFO: `/v3/users/d1381b81b5e14cfab800d6d3e8214f00` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:19.848150 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38262d13-f7b8-4cf1-8d42-fe092ef73b10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1381b81b5e14cfab800d6d3e8214f00)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:19.856719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38262d13-f7b8-4cf1-8d42-fe092ef73b10 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:19.920512 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:20.162583 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-38262d13-f7b8-4cf1-8d42-fe092ef73b10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1381b81b5e14cfab800d6d3e8214f00'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:20.185711 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-38262d13-f7b8-4cf1-8d42-fe092ef73b10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1381b81b5e14cfab800d6d3e8214f00'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:20.187129 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-38262d13-f7b8-4cf1-8d42-fe092ef73b10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1381b81b5e14cfab800d6d3e8214f00'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:20.280144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-38262d13-f7b8-4cf1-8d42-fe092ef73b10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1381b81b5e14cfab800d6d3e8214f00'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:20.303576 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-38262d13-f7b8-4cf1-8d42-fe092ef73b10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1381b81b5e14cfab800d6d3e8214f00'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:20.317788 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1142/2306] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:11:19 2022] DELETE /identity/v3/users/d1381b81b5e14cfab800d6d3e8214f00 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:20.692855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4bf4e-b374-4563-aa03-91857c9f4b02 service swift] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:20.722150 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4bf4e-b374-4563-aa03-91857c9f4b02 service swift] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:20.789521 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:20.845487 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930924cf-67b8-4ea2-a9a3-ab1515318b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:20.849731 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930924cf-67b8-4ea2-a9a3-ab1515318b2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:20.850544 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930924cf-67b8-4ea2-a9a3-ab1515318b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:20.851066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930924cf-67b8-4ea2-a9a3-ab1515318b2a admin admin] PATH_INFO: `/v3/projects/3096d9a95111415d893c6fc82ce860c4` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:20.856958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930924cf-67b8-4ea2-a9a3-ab1515318b2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3096d9a95111415d893c6fc82ce860c4)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:20.859512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930924cf-67b8-4ea2-a9a3-ab1515318b2a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:21.063794 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-930924cf-67b8-4ea2-a9a3-ab1515318b2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3096d9a95111415d893c6fc82ce860c4', 'request_id': 'req-930924cf-67b8-4ea2-a9a3-ab1515318b2a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:21.071776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-930924cf-67b8-4ea2-a9a3-ab1515318b2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3096d9a95111415d893c6fc82ce860c4', 'request_id': 'req-930924cf-67b8-4ea2-a9a3-ab1515318b2a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:21.154546 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-930924cf-67b8-4ea2-a9a3-ab1515318b2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3096d9a95111415d893c6fc82ce860c4', 'request_id': 'req-930924cf-67b8-4ea2-a9a3-ab1515318b2a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:21.172632 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-930924cf-67b8-4ea2-a9a3-ab1515318b2a admin admin] The token cache is being invalidate because project 3096d9a95111415d893c6fc82ce860c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:11:21.173033 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-930924cf-67b8-4ea2-a9a3-ab1515318b2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:21.174111 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-930924cf-67b8-4ea2-a9a3-ab1515318b2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:21.186884 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bf6de-a793-4047-bbfa-250a2b0efe56 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:21.187464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bf6de-a793-4047-bbfa-250a2b0efe56 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:21.187464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521bf6de-a793-4047-bbfa-250a2b0efe56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:21.191467 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1165/2307] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:11:20 2022] DELETE /identity/v3/projects/3096d9a95111415d893c6fc82ce860c4 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:21.223870 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-521bf6de-a793-4047-bbfa-250a2b0efe56 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:21.225415 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-521bf6de-a793-4047-bbfa-250a2b0efe56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:21.348669 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1143/2308] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:21.357348 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe67af-5f81-45dd-9e0b-c36050ee4866 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:21.357849 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe67af-5f81-45dd-9e0b-c36050ee4866 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:21.358249 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe67af-5f81-45dd-9e0b-c36050ee4866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:21.385312 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-47fe67af-5f81-45dd-9e0b-c36050ee4866 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:21.386584 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-47fe67af-5f81-45dd-9e0b-c36050ee4866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:21.403274 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1166/2309] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:11:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:21.407525 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bf6de-a793-4047-bbfa-250a2b0efe56 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:21.419470 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521bf6de-a793-4047-bbfa-250a2b0efe56 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:21.424795 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc41b3e-bc30-44b2-9ec1-42fc380dda74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:21.425423 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc41b3e-bc30-44b2-9ec1-42fc380dda74 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:21.425709 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc41b3e-bc30-44b2-9ec1-42fc380dda74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:21.426021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc41b3e-bc30-44b2-9ec1-42fc380dda74 admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:21.427289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc41b3e-bc30-44b2-9ec1-42fc380dda74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:11:21.427665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc41b3e-bc30-44b2-9ec1-42fc380dda74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:21.429234 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc41b3e-bc30-44b2-9ec1-42fc380dda74 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:21.443385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1144/2310] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:11:21.447741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe67af-5f81-45dd-9e0b-c36050ee4866 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:21.459823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe67af-5f81-45dd-9e0b-c36050ee4866 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:21.468116 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59280043-0b14-48e3-827d-acbd7b339c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:21.469023 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59280043-0b14-48e3-827d-acbd7b339c8e admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:21.469516 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59280043-0b14-48e3-827d-acbd7b339c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:21.470099 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59280043-0b14-48e3-827d-acbd7b339c8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:21.471494 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59280043-0b14-48e3-827d-acbd7b339c8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:21.474521 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59280043-0b14-48e3-827d-acbd7b339c8e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:21.592269 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1167/2311] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:11:21 2022] POST /identity/v3/projects => generated 369 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:11:21.596166 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc41b3e-bc30-44b2-9ec1-42fc380dda74 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:21.608360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc41b3e-bc30-44b2-9ec1-42fc380dda74 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:21.702914 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296740b-1ef0-465e-9c8e-d52365baf0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:21.704233 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296740b-1ef0-465e-9c8e-d52365baf0a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:21.704233 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296740b-1ef0-465e-9c8e-d52365baf0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:21.704233 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296740b-1ef0-465e-9c8e-d52365baf0a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:21.705154 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296740b-1ef0-465e-9c8e-d52365baf0a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:21.708190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296740b-1ef0-465e-9c8e-d52365baf0a6 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:21.804293 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1145/2312] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:11:21 2022] POST /identity/v3/users => generated 285 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:21.808969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59280043-0b14-48e3-827d-acbd7b339c8e admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:21.822502 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59280043-0b14-48e3-827d-acbd7b339c8e admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:21.828658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699c131-b30c-4f95-bd4f-a2d05de1d094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:21.835067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699c131-b30c-4f95-bd4f-a2d05de1d094 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:21.835067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699c131-b30c-4f95-bd4f-a2d05de1d094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:21.835067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699c131-b30c-4f95-bd4f-a2d05de1d094 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:21.835067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699c131-b30c-4f95-bd4f-a2d05de1d094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:21.835067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699c131-b30c-4f95-bd4f-a2d05de1d094 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:21.846762 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1168/2313] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:21.850585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296740b-1ef0-465e-9c8e-d52365baf0a6 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:21.864533 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296740b-1ef0-465e-9c8e-d52365baf0a6 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:21.879536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b662d61-1f04-45c6-b676-73468e506972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:21.882714 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b662d61-1f04-45c6-b676-73468e506972 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:21.883232 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b662d61-1f04-45c6-b676-73468e506972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:21.884323 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b662d61-1f04-45c6-b676-73468e506972 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:21.885657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b662d61-1f04-45c6-b676-73468e506972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:21.887471 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b662d61-1f04-45c6-b676-73468e506972 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:21.905803 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1146/2314] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:21.908387 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699c131-b30c-4f95-bd4f-a2d05de1d094 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:21.923498 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699c131-b30c-4f95-bd4f-a2d05de1d094 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:21.932295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6649049-b1cb-4128-a5b1-bf6a8c21fc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:21.933604 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6649049-b1cb-4128-a5b1-bf6a8c21fc41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:21.934188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6649049-b1cb-4128-a5b1-bf6a8c21fc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:21.934651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6649049-b1cb-4128-a5b1-bf6a8c21fc41 admin admin] PATH_INFO: `/v3/projects/f02149fc8b38412c9e4a97e80cb3a0aa/users/8e3c7f0ca3ae46e585c2de696936a2be/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:21.954115 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6649049-b1cb-4128-a5b1-bf6a8c21fc41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f02149fc8b38412c9e4a97e80cb3a0aa, user_id=8e3c7f0ca3ae46e585c2de696936a2be, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:21.958023 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6649049-b1cb-4128-a5b1-bf6a8c21fc41 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:22.022524 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1169/2315] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:11:21 2022] PUT /identity/v3/projects/f02149fc8b38412c9e4a97e80cb3a0aa/users/8e3c7f0ca3ae46e585c2de696936a2be/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:22.035543 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40fcbd8-a79b-4679-bff8-d8529ba8c85f None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:22.036096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40fcbd8-a79b-4679-bff8-d8529ba8c85f None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:22.036652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40fcbd8-a79b-4679-bff8-d8529ba8c85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:22.057194 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:22.080301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-e40fcbd8-a79b-4679-bff8-d8529ba8c85f None None] MFA Rules not processed for user `8e3c7f0ca3ae46e585c2de696936a2be`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:22.081930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e40fcbd8-a79b-4679-bff8-d8529ba8c85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:22.180092 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1147/2316] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:11:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:11:22.196659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6649049-b1cb-4128-a5b1-bf6a8c21fc41 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:22.198632 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d6649049-b1cb-4128-a5b1-bf6a8c21fc41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:22.283584 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6649049-b1cb-4128-a5b1-bf6a8c21fc41 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:22.308421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025034c-d214-4dec-badf-b49045958cb2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4abd1512ad224da896c6f2257f9602e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:22.312270 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025034c-d214-4dec-badf-b49045958cb2 service glance] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:22.312270 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025034c-d214-4dec-badf-b49045958cb2 service glance] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:22.312270 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4025034c-d214-4dec-badf-b49045958cb2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:22.342265 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:22.345769 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025034c-d214-4dec-badf-b49045958cb2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:22.347946 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4025034c-d214-4dec-badf-b49045958cb2 service glance] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:22.367048 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1170/2317] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:11:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:27.007507 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40fcbd8-a79b-4679-bff8-d8529ba8c85f None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:27.009381 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e40fcbd8-a79b-4679-bff8-d8529ba8c85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:27.082292 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40fcbd8-a79b-4679-bff8-d8529ba8c85f None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:27.087369 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976e367-9b6c-4bdf-a0ef-06e68c1a9b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:27.088019 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976e367-9b6c-4bdf-a0ef-06e68c1a9b5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:27.088303 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976e367-9b6c-4bdf-a0ef-06e68c1a9b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:27.088585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976e367-9b6c-4bdf-a0ef-06e68c1a9b5e admin admin] PATH_INFO: `/v3/users/db84b8a008ef4f46964c336a0ce87c43` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:27.089799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976e367-9b6c-4bdf-a0ef-06e68c1a9b5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=db84b8a008ef4f46964c336a0ce87c43)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:27.092732 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976e367-9b6c-4bdf-a0ef-06e68c1a9b5e admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:27.215532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025034c-d214-4dec-badf-b49045958cb2 service glance] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:27.228442 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4025034c-d214-4dec-badf-b49045958cb2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:27.292585 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4025034c-d214-4dec-badf-b49045958cb2 service glance] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:27.292585 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545cb270-2794-4a86-b449-7bf8b787a591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:27.292585 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545cb270-2794-4a86-b449-7bf8b787a591 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:27.292585 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545cb270-2794-4a86-b449-7bf8b787a591 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:27.293079 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545cb270-2794-4a86-b449-7bf8b787a591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:27.293079 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-545cb270-2794-4a86-b449-7bf8b787a591 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:27.355432 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545cb270-2794-4a86-b449-7bf8b787a591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:27.356911 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545cb270-2794-4a86-b449-7bf8b787a591 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:27.369025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1171/2318] 10.222.0.46 () {60 vars in 1396 bytes} [Tue Aug 2 09:11:27 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:27.674088 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-1976e367-9b6c-4bdf-a0ef-06e68c1a9b5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db84b8a008ef4f46964c336a0ce87c43'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:27.685439 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-1976e367-9b6c-4bdf-a0ef-06e68c1a9b5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db84b8a008ef4f46964c336a0ce87c43'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:27.686374 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-1976e367-9b6c-4bdf-a0ef-06e68c1a9b5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db84b8a008ef4f46964c336a0ce87c43'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:27.787897 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-1976e367-9b6c-4bdf-a0ef-06e68c1a9b5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db84b8a008ef4f46964c336a0ce87c43'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:27.808033 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-1976e367-9b6c-4bdf-a0ef-06e68c1a9b5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db84b8a008ef4f46964c336a0ce87c43'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:27.821923 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1148/2319] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:11:27 2022] DELETE /identity/v3/users/db84b8a008ef4f46964c336a0ce87c43 => generated 0 bytes in 813 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:27.824741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52aea6-1ecb-44f9-91a2-9f38e8947fbb None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:27.825036 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52aea6-1ecb-44f9-91a2-9f38e8947fbb None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:27.825315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52aea6-1ecb-44f9-91a2-9f38e8947fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:27.853829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-3b52aea6-1ecb-44f9-91a2-9f38e8947fbb None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:27.855415 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3b52aea6-1ecb-44f9-91a2-9f38e8947fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:27.932954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1172/2320] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:11:27.943012 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976e367-9b6c-4bdf-a0ef-06e68c1a9b5e admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:27.945876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-1976e367-9b6c-4bdf-a0ef-06e68c1a9b5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:27.968357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:27.994744 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52aea6-1ecb-44f9-91a2-9f38e8947fbb None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:27.997185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3b52aea6-1ecb-44f9-91a2-9f38e8947fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:28.022101 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976e367-9b6c-4bdf-a0ef-06e68c1a9b5e admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:28.026316 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7b55a-584d-429c-8654-37a7bffe583e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:28.040298 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7b55a-584d-429c-8654-37a7bffe583e service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:28.040298 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7b55a-584d-429c-8654-37a7bffe583e service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:28.040298 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7b55a-584d-429c-8654-37a7bffe583e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:28.040298 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7b55a-584d-429c-8654-37a7bffe583e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:28.040298 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7b55a-584d-429c-8654-37a7bffe583e service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:28.052368 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1149/2321] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:11:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:28.081735 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:28.110955 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52aea6-1ecb-44f9-91a2-9f38e8947fbb None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:28.117486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235ca266-493a-462c-a312-586e9fe6d6cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:28.118406 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ca266-493a-462c-a312-586e9fe6d6cb service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:28.118702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ca266-493a-462c-a312-586e9fe6d6cb service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:28.119301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235ca266-493a-462c-a312-586e9fe6d6cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:28.121472 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-235ca266-493a-462c-a312-586e9fe6d6cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:28.203876 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ca266-493a-462c-a312-586e9fe6d6cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:28.205795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235ca266-493a-462c-a312-586e9fe6d6cb service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:28.218190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1173/2322] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:11:27 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:28.644529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7b55a-584d-429c-8654-37a7bffe583e service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:28.646621 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-66c7b55a-584d-429c-8654-37a7bffe583e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:28.670311 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7b55a-584d-429c-8654-37a7bffe583e service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:28.679746 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691804d-1f73-4b7b-abeb-5b11cf36a300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:28.681263 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691804d-1f73-4b7b-abeb-5b11cf36a300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:28.682120 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691804d-1f73-4b7b-abeb-5b11cf36a300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:28.682120 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691804d-1f73-4b7b-abeb-5b11cf36a300 admin admin] PATH_INFO: `/v3/projects/828d7ba5e0394b3da1bb3ad9b6ca37c6` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:28.683736 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691804d-1f73-4b7b-abeb-5b11cf36a300 admin admin] RBAC: Authorizing `identity:delete_project(project_id=828d7ba5e0394b3da1bb3ad9b6ca37c6)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:28.687493 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691804d-1f73-4b7b-abeb-5b11cf36a300 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:28.860251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-7691804d-1f73-4b7b-abeb-5b11cf36a300 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '828d7ba5e0394b3da1bb3ad9b6ca37c6', 'request_id': 'req-7691804d-1f73-4b7b-abeb-5b11cf36a300'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:28.868065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-7691804d-1f73-4b7b-abeb-5b11cf36a300 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '828d7ba5e0394b3da1bb3ad9b6ca37c6', 'request_id': 'req-7691804d-1f73-4b7b-abeb-5b11cf36a300'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:28.880711 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-7691804d-1f73-4b7b-abeb-5b11cf36a300 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '828d7ba5e0394b3da1bb3ad9b6ca37c6', 'request_id': 'req-7691804d-1f73-4b7b-abeb-5b11cf36a300'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:28.980348 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-7691804d-1f73-4b7b-abeb-5b11cf36a300 admin admin] The token cache is being invalidate because project 828d7ba5e0394b3da1bb3ad9b6ca37c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:11:28.980650 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-7691804d-1f73-4b7b-abeb-5b11cf36a300 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:28.981297 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-7691804d-1f73-4b7b-abeb-5b11cf36a300 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:28.986782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1150/2323] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:11:28 2022] DELETE /identity/v3/projects/828d7ba5e0394b3da1bb3ad9b6ca37c6 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:28.991880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305808a2-294e-4e72-89c8-4023bff022a1 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:28.992108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305808a2-294e-4e72-89c8-4023bff022a1 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:28.992567 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305808a2-294e-4e72-89c8-4023bff022a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:29.022543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-305808a2-294e-4e72-89c8-4023bff022a1 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:29.023843 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-305808a2-294e-4e72-89c8-4023bff022a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:29.099307 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1174/2324] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:11:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:11:29.108552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcc550-3cb9-4f72-ae8e-618cf4a9477f None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:29.109071 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcc550-3cb9-4f72-ae8e-618cf4a9477f None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:29.109164 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbcc550-3cb9-4f72-ae8e-618cf4a9477f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:29.136946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-cdbcc550-3cb9-4f72-ae8e-618cf4a9477f None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:29.138248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-cdbcc550-3cb9-4f72-ae8e-618cf4a9477f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:29.150619 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1151/2325] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:11:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:29.155739 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305808a2-294e-4e72-89c8-4023bff022a1 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:29.166671 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305808a2-294e-4e72-89c8-4023bff022a1 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:29.173770 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41d68c-ff1c-40be-8896-f88fb5e7dbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:29.174513 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41d68c-ff1c-40be-8896-f88fb5e7dbae admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:29.174916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41d68c-ff1c-40be-8896-f88fb5e7dbae admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:29.175265 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e41d68c-ff1c-40be-8896-f88fb5e7dbae admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:29.176052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e41d68c-ff1c-40be-8896-f88fb5e7dbae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:11:29.176522 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e41d68c-ff1c-40be-8896-f88fb5e7dbae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:29.178282 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e41d68c-ff1c-40be-8896-f88fb5e7dbae admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:29.194014 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1175/2326] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:11:29.196783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbcc550-3cb9-4f72-ae8e-618cf4a9477f None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:29.214890 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbcc550-3cb9-4f72-ae8e-618cf4a9477f None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:29.221628 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4373a8-3e8a-457e-bef5-4583637039d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:29.221824 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4373a8-3e8a-457e-bef5-4583637039d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:29.222165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4373a8-3e8a-457e-bef5-4583637039d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:29.222544 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4373a8-3e8a-457e-bef5-4583637039d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:29.223555 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4373a8-3e8a-457e-bef5-4583637039d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:29.225404 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4373a8-3e8a-457e-bef5-4583637039d8 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:29.355839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41d68c-ff1c-40be-8896-f88fb5e7dbae admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:29.356708 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1152/2327] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:11:29 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:11:29.368247 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e41d68c-ff1c-40be-8896-f88fb5e7dbae admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:29.458814 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4fb60-e4ba-45ca-96da-d6b3e30f8414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:29.459225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4fb60-e4ba-45ca-96da-d6b3e30f8414 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:29.459551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4fb60-e4ba-45ca-96da-d6b3e30f8414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:29.459788 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4fb60-e4ba-45ca-96da-d6b3e30f8414 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:29.460727 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4fb60-e4ba-45ca-96da-d6b3e30f8414 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:29.462699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4fb60-e4ba-45ca-96da-d6b3e30f8414 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:29.585012 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1176/2328] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:11:29 2022] POST /identity/v3/users => generated 287 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:11:29.589859 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4373a8-3e8a-457e-bef5-4583637039d8 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:29.607486 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4373a8-3e8a-457e-bef5-4583637039d8 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:29.623924 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf9da3-fc69-4ab3-930e-5664f29f3e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:29.624567 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf9da3-fc69-4ab3-930e-5664f29f3e00 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:29.624852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf9da3-fc69-4ab3-930e-5664f29f3e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:29.625131 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf9da3-fc69-4ab3-930e-5664f29f3e00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:29.625834 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf9da3-fc69-4ab3-930e-5664f29f3e00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:29.627551 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf9da3-fc69-4ab3-930e-5664f29f3e00 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:29.660386 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1153/2329] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:29.663611 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4fb60-e4ba-45ca-96da-d6b3e30f8414 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:29.671587 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:29.680044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4fb60-e4ba-45ca-96da-d6b3e30f8414 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:29.691635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfcaa1f-e95e-48e5-bb94-c53e732cba25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:29.693311 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfcaa1f-e95e-48e5-bb94-c53e732cba25 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:29.693834 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfcaa1f-e95e-48e5-bb94-c53e732cba25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:29.694360 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfcaa1f-e95e-48e5-bb94-c53e732cba25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:29.696018 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfcaa1f-e95e-48e5-bb94-c53e732cba25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:29.697925 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfcaa1f-e95e-48e5-bb94-c53e732cba25 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:29.723612 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf9da3-fc69-4ab3-930e-5664f29f3e00 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:29.728890 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1177/2330] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:11:29 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:11:29.739974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf9da3-fc69-4ab3-930e-5664f29f3e00 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:29.746607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baff01c-98c5-4038-961b-613e731c5a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:29.747589 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baff01c-98c5-4038-961b-613e731c5a2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:29.747973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baff01c-98c5-4038-961b-613e731c5a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:29.748356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baff01c-98c5-4038-961b-613e731c5a2a admin admin] PATH_INFO: `/v3/projects/bb910e56eeb74717967e1c6e52a28943/users/521f8aef043f4a1a93929f2317f369b1/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:29.773783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baff01c-98c5-4038-961b-613e731c5a2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb910e56eeb74717967e1c6e52a28943, user_id=521f8aef043f4a1a93929f2317f369b1, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:29.776587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baff01c-98c5-4038-961b-613e731c5a2a admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:29.872229 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1154/2331] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:11:29 2022] PUT /identity/v3/projects/bb910e56eeb74717967e1c6e52a28943/users/521f8aef043f4a1a93929f2317f369b1/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:29.883883 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560ecb5-3a4c-4b2c-bf57-5eeaa77c23d4 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:29.884174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560ecb5-3a4c-4b2c-bf57-5eeaa77c23d4 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:29.884455 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5560ecb5-3a4c-4b2c-bf57-5eeaa77c23d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:29.920295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5560ecb5-3a4c-4b2c-bf57-5eeaa77c23d4 None None] MFA Rules not processed for user `521f8aef043f4a1a93929f2317f369b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:29.926351 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5560ecb5-3a4c-4b2c-bf57-5eeaa77c23d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:30.009486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1178/2332] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:11:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:30.022517 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baff01c-98c5-4038-961b-613e731c5a2a admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:30.028144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-7baff01c-98c5-4038-961b-613e731c5a2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:30.099624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:30.120910 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baff01c-98c5-4038-961b-613e731c5a2a admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:30.127772 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bee315-646d-4de9-8046-089a6475ecb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:30.128945 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bee315-646d-4de9-8046-089a6475ecb0 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:30.129226 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bee315-646d-4de9-8046-089a6475ecb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:30.129498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bee315-646d-4de9-8046-089a6475ecb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:30.142596 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bee315-646d-4de9-8046-089a6475ecb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:30.144794 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bee315-646d-4de9-8046-089a6475ecb0 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:30.159887 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1155/2333] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:11:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:11:30.191239 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560ecb5-3a4c-4b2c-bf57-5eeaa77c23d4 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:30.276596 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5560ecb5-3a4c-4b2c-bf57-5eeaa77c23d4 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:30.301403 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de149769-48fc-49c2-b0b2-35f667839151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:30.302427 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de149769-48fc-49c2-b0b2-35f667839151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:30.302897 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de149769-48fc-49c2-b0b2-35f667839151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:30.303341 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de149769-48fc-49c2-b0b2-35f667839151 admin admin] PATH_INFO: `/v3/users/521f8aef043f4a1a93929f2317f369b1` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:30.306743 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de149769-48fc-49c2-b0b2-35f667839151 admin admin] RBAC: Authorizing `identity:delete_user(user_id=521f8aef043f4a1a93929f2317f369b1)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:30.311640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de149769-48fc-49c2-b0b2-35f667839151 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:30.523277 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-de149769-48fc-49c2-b0b2-35f667839151 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '521f8aef043f4a1a93929f2317f369b1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:30.535028 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-de149769-48fc-49c2-b0b2-35f667839151 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '521f8aef043f4a1a93929f2317f369b1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:30.536146 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-de149769-48fc-49c2-b0b2-35f667839151 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '521f8aef043f4a1a93929f2317f369b1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:30.558720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-de149769-48fc-49c2-b0b2-35f667839151 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '521f8aef043f4a1a93929f2317f369b1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:30.559608 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-de149769-48fc-49c2-b0b2-35f667839151 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '521f8aef043f4a1a93929f2317f369b1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:30.655735 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1179/2334] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:11:30 2022] DELETE /identity/v3/users/521f8aef043f4a1a93929f2317f369b1 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:30.661205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ef682-edd4-4d84-aefe-22b1768fd707 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:30.661494 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ef682-edd4-4d84-aefe-22b1768fd707 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:30.661775 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61ef682-edd4-4d84-aefe-22b1768fd707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:30.685487 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-e61ef682-edd4-4d84-aefe-22b1768fd707 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:30.686832 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e61ef682-edd4-4d84-aefe-22b1768fd707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:30.833867 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1156/2335] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:11:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:11:30.890127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de149769-48fc-49c2-b0b2-35f667839151 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:30.891658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-de149769-48fc-49c2-b0b2-35f667839151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:30.897734 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:30.993404 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de149769-48fc-49c2-b0b2-35f667839151 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:30.997579 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73e8d5-98de-4668-8cc5-40224fd8789b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:30.998352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73e8d5-98de-4668-8cc5-40224fd8789b service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:30.998680 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73e8d5-98de-4668-8cc5-40224fd8789b service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:30.999096 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f73e8d5-98de-4668-8cc5-40224fd8789b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:31.010384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f73e8d5-98de-4668-8cc5-40224fd8789b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:31.019422 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f73e8d5-98de-4668-8cc5-40224fd8789b service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:31.027072 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1180/2336] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:11:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:31.724459 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ef682-edd4-4d84-aefe-22b1768fd707 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:31.725963 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e61ef682-edd4-4d84-aefe-22b1768fd707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:31.751218 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61ef682-edd4-4d84-aefe-22b1768fd707 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:31.767963 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e02e70-f386-47a3-b6b0-d9bbcc0b8779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:31.768671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e02e70-f386-47a3-b6b0-d9bbcc0b8779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:31.768959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e02e70-f386-47a3-b6b0-d9bbcc0b8779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:31.769238 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e02e70-f386-47a3-b6b0-d9bbcc0b8779 admin admin] PATH_INFO: `/v3/projects/bb910e56eeb74717967e1c6e52a28943` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:31.770598 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e02e70-f386-47a3-b6b0-d9bbcc0b8779 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb910e56eeb74717967e1c6e52a28943)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:31.772665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e02e70-f386-47a3-b6b0-d9bbcc0b8779 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:31.953691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-91e02e70-f386-47a3-b6b0-d9bbcc0b8779 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb910e56eeb74717967e1c6e52a28943', 'request_id': 'req-91e02e70-f386-47a3-b6b0-d9bbcc0b8779'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:31.963696 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-91e02e70-f386-47a3-b6b0-d9bbcc0b8779 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb910e56eeb74717967e1c6e52a28943', 'request_id': 'req-91e02e70-f386-47a3-b6b0-d9bbcc0b8779'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:31.974633 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-91e02e70-f386-47a3-b6b0-d9bbcc0b8779 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb910e56eeb74717967e1c6e52a28943', 'request_id': 'req-91e02e70-f386-47a3-b6b0-d9bbcc0b8779'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:32.051448 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-91e02e70-f386-47a3-b6b0-d9bbcc0b8779 admin admin] The token cache is being invalidate because project bb910e56eeb74717967e1c6e52a28943 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:11:32.051747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-91e02e70-f386-47a3-b6b0-d9bbcc0b8779 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:32.053783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-91e02e70-f386-47a3-b6b0-d9bbcc0b8779 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:11:32.056404 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1157/2337] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:11:31 2022] DELETE /identity/v3/projects/bb910e56eeb74717967e1c6e52a28943 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:11:40.684353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73e8d5-98de-4668-8cc5-40224fd8789b service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:40.685842 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5f73e8d5-98de-4668-8cc5-40224fd8789b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:40.698308 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f73e8d5-98de-4668-8cc5-40224fd8789b service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:40.752643 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b6551-7f11-444f-9244-096fbc7ff214 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:40.753271 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b6551-7f11-444f-9244-096fbc7ff214 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:40.753554 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b6551-7f11-444f-9244-096fbc7ff214 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:40.753830 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b6551-7f11-444f-9244-096fbc7ff214 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:40.755636 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3f0b6551-7f11-444f-9244-096fbc7ff214 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:40.765760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b6551-7f11-444f-9244-096fbc7ff214 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:40.767249 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b6551-7f11-444f-9244-096fbc7ff214 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:40.776590 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1181/2338] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:11:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:11:41.155054 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffa66e-39b5-4926-908f-d5f2aa1ea498 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:41.155588 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffa66e-39b5-4926-908f-d5f2aa1ea498 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:41.155588 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffa66e-39b5-4926-908f-d5f2aa1ea498 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:41.156960 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1158/2339] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 2 09:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:11:41.163718 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43876a3-c144-4a76-aada-9ba7d691de9b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:41.164354 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43876a3-c144-4a76-aada-9ba7d691de9b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:41.164354 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43876a3-c144-4a76-aada-9ba7d691de9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:41.187020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-b43876a3-c144-4a76-aada-9ba7d691de9b None None] MFA Rules not processed for user `3191d804e8dc4b1a85f490319833ec07`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:41.188254 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b43876a3-c144-4a76-aada-9ba7d691de9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:41.262518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1182/2340] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 2 09:11:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:41.274479 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cffa66e-39b5-4926-908f-d5f2aa1ea498 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:41.276148 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-2cffa66e-39b5-4926-908f-d5f2aa1ea498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:41.342152 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cffa66e-39b5-4926-908f-d5f2aa1ea498 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:41.345718 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452aad4-8eca-411e-bd21-458346579bdc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4abd1512ad224da896c6f2257f9602e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:41.346388 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452aad4-8eca-411e-bd21-458346579bdc service glance] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:41.346665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452aad4-8eca-411e-bd21-458346579bdc service glance] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:41.346936 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452aad4-8eca-411e-bd21-458346579bdc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:41.358148 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452aad4-8eca-411e-bd21-458346579bdc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:41.359673 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452aad4-8eca-411e-bd21-458346579bdc service glance] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:41.371612 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1159/2341] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:11:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:41.608959 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b29eb-06bc-4f46-8320-e57179c699fe None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:41.609256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b29eb-06bc-4f46-8320-e57179c699fe None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:41.609570 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726b29eb-06bc-4f46-8320-e57179c699fe None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:41.610961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1183/2342] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 2 09:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:11:41.616878 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91321bfb-06d1-4c88-acee-88cefb30e7e3 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:41.617192 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91321bfb-06d1-4c88-acee-88cefb30e7e3 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:41.617467 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91321bfb-06d1-4c88-acee-88cefb30e7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:41.633428 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:41.639058 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-91321bfb-06d1-4c88-acee-88cefb30e7e3 None None] MFA Rules not processed for user `ddf6f4d84bb34d2f810a0f6cec36da4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:41.640325 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-91321bfb-06d1-4c88-acee-88cefb30e7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:41.647092 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1160/2343] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 2 09:11:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:41.659581 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b29eb-06bc-4f46-8320-e57179c699fe None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:41.661367 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-726b29eb-06bc-4f46-8320-e57179c699fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:41.676830 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726b29eb-06bc-4f46-8320-e57179c699fe None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:41.680530 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89d76b-d037-40fe-be60-a48de27a8162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:41.681163 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89d76b-d037-40fe-be60-a48de27a8162 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:41.681445 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89d76b-d037-40fe-be60-a48de27a8162 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:41.681720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89d76b-d037-40fe-be60-a48de27a8162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:41.690699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89d76b-d037-40fe-be60-a48de27a8162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:41.692186 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89d76b-d037-40fe-be60-a48de27a8162 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:41.701838 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1184/2344] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:11:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:41.790672 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a969ce-33dd-4428-b0ea-3a6233588295 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:41.790958 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a969ce-33dd-4428-b0ea-3a6233588295 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:41.791233 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a969ce-33dd-4428-b0ea-3a6233588295 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:41.792700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1161/2345] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 2 09:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:11:41.797676 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed91761-cd89-4227-85e8-eaced8339e88 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:41.798020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed91761-cd89-4227-85e8-eaced8339e88 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:41.798292 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed91761-cd89-4227-85e8-eaced8339e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:41.819327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9ed91761-cd89-4227-85e8-eaced8339e88 None None] MFA Rules not processed for user `3191d804e8dc4b1a85f490319833ec07`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:41.820840 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9ed91761-cd89-4227-85e8-eaced8339e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:41.830241 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1185/2346] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 2 09:11:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:41.840811 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a969ce-33dd-4428-b0ea-3a6233588295 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:41.851995 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a969ce-33dd-4428-b0ea-3a6233588295 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:41.855735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737cebd-4a0d-4ad7-b359-de530de0fdf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:41.856343 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737cebd-4a0d-4ad7-b359-de530de0fdf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:41.856620 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737cebd-4a0d-4ad7-b359-de530de0fdf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:41.856893 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737cebd-4a0d-4ad7-b359-de530de0fdf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:41.868434 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737cebd-4a0d-4ad7-b359-de530de0fdf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:41.869867 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737cebd-4a0d-4ad7-b359-de530de0fdf5 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:41.881990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1162/2347] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:11:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:41.945869 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed91761-cd89-4227-85e8-eaced8339e88 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:41.947299 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9ed91761-cd89-4227-85e8-eaced8339e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:41.969104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:42.019857 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed91761-cd89-4227-85e8-eaced8339e88 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:42.025059 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad456ca-e46d-4599-86dc-f15706d7a762 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad5cc9927374836b0cc74db85b686fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:42.025695 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad456ca-e46d-4599-86dc-f15706d7a762 service placement] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:42.026054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad456ca-e46d-4599-86dc-f15706d7a762 service placement] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:42.026334 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad456ca-e46d-4599-86dc-f15706d7a762 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:42.028210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2ad456ca-e46d-4599-86dc-f15706d7a762 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:42.042262 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad456ca-e46d-4599-86dc-f15706d7a762 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:42.043886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad456ca-e46d-4599-86dc-f15706d7a762 service placement] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:42.055687 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1186/2348] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:11:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:48.178264 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737cebd-4a0d-4ad7-b359-de530de0fdf5 service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:48.188606 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737cebd-4a0d-4ad7-b359-de530de0fdf5 service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:48.193430 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a6d9c-7546-4a1e-bbc9-58601363afa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:48.194009 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a6d9c-7546-4a1e-bbc9-58601363afa2 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:48.194293 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a6d9c-7546-4a1e-bbc9-58601363afa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:48.194568 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a6d9c-7546-4a1e-bbc9-58601363afa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:48.196957 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-a20a6d9c-7546-4a1e-bbc9-58601363afa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:48.208652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a6d9c-7546-4a1e-bbc9-58601363afa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:48.210643 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20a6d9c-7546-4a1e-bbc9-58601363afa2 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:48.220812 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1163/2349] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:11:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:11:52.050983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=23 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:11:56.395930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84cf528-bd20-42df-8be3-eaa83f624ea4 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:56.396484 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84cf528-bd20-42df-8be3-eaa83f624ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:56.396484 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84cf528-bd20-42df-8be3-eaa83f624ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:56.417520 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f84cf528-bd20-42df-8be3-eaa83f624ea4 None None] MFA Rules not processed for user `ed8114b7a28741268fdff71c866dbc77`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:11:56.418789 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f84cf528-bd20-42df-8be3-eaa83f624ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:56.483124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1187/2350] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 2 09:11:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:11:56.495989 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a6d9c-7546-4a1e-bbc9-58601363afa2 service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:11:56.497445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-a20a6d9c-7546-4a1e-bbc9-58601363afa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:11:56.580656 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a6d9c-7546-4a1e-bbc9-58601363afa2 service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:11:56.590901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca013f83-27db-4b63-b2ee-c5dacdb9239e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff92ef3b9da74ccab3fbc43f852e3ad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:11:56.590901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca013f83-27db-4b63-b2ee-c5dacdb9239e service swift] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:11:56.590901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca013f83-27db-4b63-b2ee-c5dacdb9239e service swift] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:11:56.590901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca013f83-27db-4b63-b2ee-c5dacdb9239e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:11:56.596852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca013f83-27db-4b63-b2ee-c5dacdb9239e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:11:56.598591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca013f83-27db-4b63-b2ee-c5dacdb9239e service swift] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:11:56.608688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1164/2351] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 2 09:11:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:12:06.475362 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=25 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:06.604686 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=44 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:39.283087 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84cf528-bd20-42df-8be3-eaa83f624ea4 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:39.284695 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f84cf528-bd20-42df-8be3-eaa83f624ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:39.358787 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84cf528-bd20-42df-8be3-eaa83f624ea4 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:39.364943 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae9307-d89d-4f8a-9238-0029d852bb8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:39.365562 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae9307-d89d-4f8a-9238-0029d852bb8e service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:39.365840 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae9307-d89d-4f8a-9238-0029d852bb8e service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:39.366143 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae9307-d89d-4f8a-9238-0029d852bb8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:39.367922 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f5ae9307-d89d-4f8a-9238-0029d852bb8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:39.382094 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae9307-d89d-4f8a-9238-0029d852bb8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:39.383741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae9307-d89d-4f8a-9238-0029d852bb8e service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:39.397950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1188/2352] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:12:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:12:42.928515 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca013f83-27db-4b63-b2ee-c5dacdb9239e service swift] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:42.938411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-ca013f83-27db-4b63-b2ee-c5dacdb9239e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:43.060568 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca013f83-27db-4b63-b2ee-c5dacdb9239e service swift] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:43.065655 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd38f0-67dc-420e-9997-296d6dc39faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:43.066696 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd38f0-67dc-420e-9997-296d6dc39faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:43.066696 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd38f0-67dc-420e-9997-296d6dc39faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:43.066901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd38f0-67dc-420e-9997-296d6dc39faa admin admin] PATH_INFO: `/v3/users/8e3c7f0ca3ae46e585c2de696936a2be` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:43.068327 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd38f0-67dc-420e-9997-296d6dc39faa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e3c7f0ca3ae46e585c2de696936a2be)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:43.070249 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd38f0-67dc-420e-9997-296d6dc39faa admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:43.312655 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9acd38f0-67dc-420e-9997-296d6dc39faa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e3c7f0ca3ae46e585c2de696936a2be'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:43.322243 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae9307-d89d-4f8a-9238-0029d852bb8e service nova] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:43.323703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f5ae9307-d89d-4f8a-9238-0029d852bb8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:43.328110 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9acd38f0-67dc-420e-9997-296d6dc39faa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e3c7f0ca3ae46e585c2de696936a2be'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:43.328817 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9acd38f0-67dc-420e-9997-296d6dc39faa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e3c7f0ca3ae46e585c2de696936a2be'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:43.338279 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae9307-d89d-4f8a-9238-0029d852bb8e service nova] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:43.343804 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204896a-ee00-441b-ae0c-69848d9abd95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:43.344423 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204896a-ee00-441b-ae0c-69848d9abd95 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:43.344707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204896a-ee00-441b-ae0c-69848d9abd95 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:43.345178 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204896a-ee00-441b-ae0c-69848d9abd95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:43.346947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c204896a-ee00-441b-ae0c-69848d9abd95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:43.359249 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204896a-ee00-441b-ae0c-69848d9abd95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:43.360824 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204896a-ee00-441b-ae0c-69848d9abd95 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:43.371841 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1189/2353] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:12:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:12:43.446761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9acd38f0-67dc-420e-9997-296d6dc39faa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e3c7f0ca3ae46e585c2de696936a2be'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:43.457118 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9acd38f0-67dc-420e-9997-296d6dc39faa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e3c7f0ca3ae46e585c2de696936a2be'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:43.464736 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1165/2354] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:12:42 2022] DELETE /identity/v3/users/8e3c7f0ca3ae46e585c2de696936a2be => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:43.470316 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c04c05-e3dc-40fe-af6b-5560807adb68 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:43.470604 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c04c05-e3dc-40fe-af6b-5560807adb68 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:43.471289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c04c05-e3dc-40fe-af6b-5560807adb68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:43.500376 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-01c04c05-e3dc-40fe-af6b-5560807adb68 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:43.501707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-01c04c05-e3dc-40fe-af6b-5560807adb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:43.562118 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1190/2355] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:12:43.573831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd38f0-67dc-420e-9997-296d6dc39faa admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:43.575481 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-9acd38f0-67dc-420e-9997-296d6dc39faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:43.609298 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:43.657452 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd38f0-67dc-420e-9997-296d6dc39faa admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:43.661120 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95417a9e-daa9-4725-ad1d-c6efbf874923 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:43.661744 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95417a9e-daa9-4725-ad1d-c6efbf874923 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:43.662070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95417a9e-daa9-4725-ad1d-c6efbf874923 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:43.662426 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95417a9e-daa9-4725-ad1d-c6efbf874923 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:43.671784 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95417a9e-daa9-4725-ad1d-c6efbf874923 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:43.673210 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95417a9e-daa9-4725-ad1d-c6efbf874923 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:43.683714 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1166/2356] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:12:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:12:44.161939 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c04c05-e3dc-40fe-af6b-5560807adb68 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:44.164301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-01c04c05-e3dc-40fe-af6b-5560807adb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:44.178597 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c04c05-e3dc-40fe-af6b-5560807adb68 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:44.201945 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b1563-9ad5-4824-88eb-84a67a7a61d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:44.202590 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b1563-9ad5-4824-88eb-84a67a7a61d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:44.202878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b1563-9ad5-4824-88eb-84a67a7a61d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:44.203154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b1563-9ad5-4824-88eb-84a67a7a61d2 admin admin] PATH_INFO: `/v3/projects/f02149fc8b38412c9e4a97e80cb3a0aa` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:44.204425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b1563-9ad5-4824-88eb-84a67a7a61d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f02149fc8b38412c9e4a97e80cb3a0aa)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:44.206881 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b1563-9ad5-4824-88eb-84a67a7a61d2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:44.346309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fa8b1563-9ad5-4824-88eb-84a67a7a61d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f02149fc8b38412c9e4a97e80cb3a0aa', 'request_id': 'req-fa8b1563-9ad5-4824-88eb-84a67a7a61d2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:44.353204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fa8b1563-9ad5-4824-88eb-84a67a7a61d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f02149fc8b38412c9e4a97e80cb3a0aa', 'request_id': 'req-fa8b1563-9ad5-4824-88eb-84a67a7a61d2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:44.466408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fa8b1563-9ad5-4824-88eb-84a67a7a61d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f02149fc8b38412c9e4a97e80cb3a0aa', 'request_id': 'req-fa8b1563-9ad5-4824-88eb-84a67a7a61d2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:44.478014 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fa8b1563-9ad5-4824-88eb-84a67a7a61d2 admin admin] The token cache is being invalidate because project f02149fc8b38412c9e4a97e80cb3a0aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:12:44.478302 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fa8b1563-9ad5-4824-88eb-84a67a7a61d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:44.478929 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fa8b1563-9ad5-4824-88eb-84a67a7a61d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:12:44.480516 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1191/2357] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:12:44 2022] DELETE /identity/v3/projects/f02149fc8b38412c9e4a97e80cb3a0aa => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:44.485301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888ac1b-02f0-4ffc-80d3-c31ad7915c9c None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:44.485798 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888ac1b-02f0-4ffc-80d3-c31ad7915c9c None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:44.485929 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888ac1b-02f0-4ffc-80d3-c31ad7915c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:44.537069 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-b888ac1b-02f0-4ffc-80d3-c31ad7915c9c None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:44.538407 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-b888ac1b-02f0-4ffc-80d3-c31ad7915c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:44.597665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1167/2358] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:44.606735 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be52fb-af52-44e0-b0db-1943a94ae3b8 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:44.607050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be52fb-af52-44e0-b0db-1943a94ae3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:44.607325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be52fb-af52-44e0-b0db-1943a94ae3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:44.614939 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:44.628876 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-d8be52fb-af52-44e0-b0db-1943a94ae3b8 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:44.630411 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d8be52fb-af52-44e0-b0db-1943a94ae3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:44.638455 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1192/2359] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:44.642975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888ac1b-02f0-4ffc-80d3-c31ad7915c9c None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:44.653999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888ac1b-02f0-4ffc-80d3-c31ad7915c9c None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:44.658773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ec3e0-5aad-40c3-9342-89f3c8106743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:44.659302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ec3e0-5aad-40c3-9342-89f3c8106743 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:44.663461 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ec3e0-5aad-40c3-9342-89f3c8106743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:44.663461 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7ec3e0-5aad-40c3-9342-89f3c8106743 admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:44.663461 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ec3e0-5aad-40c3-9342-89f3c8106743 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:12:44.663461 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ec3e0-5aad-40c3-9342-89f3c8106743 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:44.663461 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7ec3e0-5aad-40c3-9342-89f3c8106743 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:44.675008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1168/2360] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:12:44.679477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be52fb-af52-44e0-b0db-1943a94ae3b8 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:44.692166 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be52fb-af52-44e0-b0db-1943a94ae3b8 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:44.694764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426dd2c-5afa-43e1-8a59-3650b48ec1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:44.695380 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426dd2c-5afa-43e1-8a59-3650b48ec1a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:44.695680 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426dd2c-5afa-43e1-8a59-3650b48ec1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:44.696002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426dd2c-5afa-43e1-8a59-3650b48ec1a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:44.696619 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426dd2c-5afa-43e1-8a59-3650b48ec1a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:44.698556 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3426dd2c-5afa-43e1-8a59-3650b48ec1a0 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:44.839615 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1193/2361] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:12:44 2022] POST /identity/v3/projects => generated 367 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:12:44.845279 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ec3e0-5aad-40c3-9342-89f3c8106743 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:44.854444 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7ec3e0-5aad-40c3-9342-89f3c8106743 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:44.911331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40086475-cd56-46ac-a69c-ef681e4f1fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:44.911953 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40086475-cd56-46ac-a69c-ef681e4f1fb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:44.912236 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40086475-cd56-46ac-a69c-ef681e4f1fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:44.912606 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40086475-cd56-46ac-a69c-ef681e4f1fb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:44.913278 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40086475-cd56-46ac-a69c-ef681e4f1fb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:44.915241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40086475-cd56-46ac-a69c-ef681e4f1fb4 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:45.021008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1169/2362] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:12:44 2022] POST /identity/v3/users => generated 284 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:12:45.025219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426dd2c-5afa-43e1-8a59-3650b48ec1a0 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:45.035699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426dd2c-5afa-43e1-8a59-3650b48ec1a0 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:45.040656 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7e7c5-de24-4575-8293-e10adc1d2771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:45.041268 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7e7c5-de24-4575-8293-e10adc1d2771 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:45.041555 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7e7c5-de24-4575-8293-e10adc1d2771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:45.041836 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7e7c5-de24-4575-8293-e10adc1d2771 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:45.042598 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b7e7c5-de24-4575-8293-e10adc1d2771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:45.044209 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b7e7c5-de24-4575-8293-e10adc1d2771 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:45.055285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1194/2363] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:12:45.059381 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40086475-cd56-46ac-a69c-ef681e4f1fb4 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:45.064522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:45.070945 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40086475-cd56-46ac-a69c-ef681e4f1fb4 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:45.075682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0e5c8-ee73-4bd3-b17f-67cc428be172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:45.078510 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0e5c8-ee73-4bd3-b17f-67cc428be172 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:45.078510 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0e5c8-ee73-4bd3-b17f-67cc428be172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:45.078510 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0e5c8-ee73-4bd3-b17f-67cc428be172 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:45.078510 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0e5c8-ee73-4bd3-b17f-67cc428be172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:45.079178 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0e5c8-ee73-4bd3-b17f-67cc428be172 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:45.090059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1170/2364] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:12:45.094172 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7e7c5-de24-4575-8293-e10adc1d2771 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:45.104082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7e7c5-de24-4575-8293-e10adc1d2771 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:45.108947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971560f-4ed2-4cde-9354-c13b6e231eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:45.109600 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971560f-4ed2-4cde-9354-c13b6e231eff admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:45.109901 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971560f-4ed2-4cde-9354-c13b6e231eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:45.110184 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971560f-4ed2-4cde-9354-c13b6e231eff admin admin] PATH_INFO: `/v3/projects/5ca601aa79704545982a0a53c4bffe3d/users/4be83520b08d4006b12b3308202447e0/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:45.125353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971560f-4ed2-4cde-9354-c13b6e231eff admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ca601aa79704545982a0a53c4bffe3d, user_id=4be83520b08d4006b12b3308202447e0, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:45.128094 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971560f-4ed2-4cde-9354-c13b6e231eff admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:45.205699 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1195/2365] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:12:45 2022] PUT /identity/v3/projects/5ca601aa79704545982a0a53c4bffe3d/users/4be83520b08d4006b12b3308202447e0/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:12:45.276369 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5af3a-6089-4f3a-947c-3de8f2712453 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:45.276593 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5af3a-6089-4f3a-947c-3de8f2712453 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:45.276825 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a5af3a-6089-4f3a-947c-3de8f2712453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:45.305042 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-22a5af3a-6089-4f3a-947c-3de8f2712453 None None] MFA Rules not processed for user `4be83520b08d4006b12b3308202447e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:45.307665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-22a5af3a-6089-4f3a-947c-3de8f2712453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:45.382878 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1171/2366] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:12:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:45.394163 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971560f-4ed2-4cde-9354-c13b6e231eff admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:45.396102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d971560f-4ed2-4cde-9354-c13b6e231eff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:45.452760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971560f-4ed2-4cde-9354-c13b6e231eff admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:45.456413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a2128-9083-4a47-a9c7-e29575c71e44 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4abd1512ad224da896c6f2257f9602e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:45.457027 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a2128-9083-4a47-a9c7-e29575c71e44 service glance] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:45.457310 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a2128-9083-4a47-a9c7-e29575c71e44 service glance] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:45.457587 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a2128-9083-4a47-a9c7-e29575c71e44 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:45.466823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a2128-9083-4a47-a9c7-e29575c71e44 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:45.468276 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a2128-9083-4a47-a9c7-e29575c71e44 service glance] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:45.478595 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1196/2367] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:12:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:12:50.537543 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f84618-6816-413c-813a-b20fff7c9119 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:50.538028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f84618-6816-413c-813a-b20fff7c9119 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:50.538194 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f84618-6816-413c-813a-b20fff7c9119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:50.569023 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-04f84618-6816-413c-813a-b20fff7c9119 None None] MFA Rules not processed for user `ed8114b7a28741268fdff71c866dbc77`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:12:50.570265 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-04f84618-6816-413c-813a-b20fff7c9119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:50.623620 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1172/2368] 10.222.0.46 () {58 vars in 967 bytes} [Tue Aug 2 09:12:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:12:50.647874 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a2128-9083-4a47-a9c7-e29575c71e44 service glance] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:50.648328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e75a2128-9083-4a47-a9c7-e29575c71e44 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:50.719880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a2128-9083-4a47-a9c7-e29575c71e44 service glance] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:50.724792 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b75854-5555-40ed-9f74-ca55fd6b356c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff92ef3b9da74ccab3fbc43f852e3ad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:50.725394 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b75854-5555-40ed-9f74-ca55fd6b356c service swift] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:50.725673 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b75854-5555-40ed-9f74-ca55fd6b356c service swift] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:50.725973 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b75854-5555-40ed-9f74-ca55fd6b356c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:50.733847 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b75854-5555-40ed-9f74-ca55fd6b356c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:50.735436 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b75854-5555-40ed-9f74-ca55fd6b356c service swift] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:50.743409 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1197/2369] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 2 09:12:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:12:56.725778 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f84618-6816-413c-813a-b20fff7c9119 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:12:56.727685 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-04f84618-6816-413c-813a-b20fff7c9119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:56.814451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f84618-6816-413c-813a-b20fff7c9119 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:12:56.822052 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46a900-974b-4722-955e-7ffaba148d59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:12:56.822887 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46a900-974b-4722-955e-7ffaba148d59 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:12:56.823175 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46a900-974b-4722-955e-7ffaba148d59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:12:56.823494 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46a900-974b-4722-955e-7ffaba148d59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:12:56.825619 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-5e46a900-974b-4722-955e-7ffaba148d59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:12:56.837939 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:12:56.890634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46a900-974b-4722-955e-7ffaba148d59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:12:56.892104 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46a900-974b-4722-955e-7ffaba148d59 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:12:56.903145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1173/2370] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:12:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:13:00.740190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=88,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:06.898328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=22 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:15.685266 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b75854-5555-40ed-9f74-ca55fd6b356c service swift] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:15.686737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c9b75854-5555-40ed-9f74-ca55fd6b356c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:15.766262 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b75854-5555-40ed-9f74-ca55fd6b356c service swift] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:15.772068 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e865a0-4eb9-49e0-842d-99a0cf6dc0c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:15.773219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e865a0-4eb9-49e0-842d-99a0cf6dc0c2 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:15.773503 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e865a0-4eb9-49e0-842d-99a0cf6dc0c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:15.773796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e865a0-4eb9-49e0-842d-99a0cf6dc0c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:15.775967 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-38e865a0-4eb9-49e0-842d-99a0cf6dc0c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:15.837969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e865a0-4eb9-49e0-842d-99a0cf6dc0c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:15.839536 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e865a0-4eb9-49e0-842d-99a0cf6dc0c2 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:15.849082 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1198/2371] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:13:15 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:13:17.932370 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46a900-974b-4722-955e-7ffaba148d59 service cinder] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:17.935483 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-5e46a900-974b-4722-955e-7ffaba148d59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:17.996979 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46a900-974b-4722-955e-7ffaba148d59 service cinder] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:18.000755 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2f823-f647-4e57-9607-12fad760c6d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:18.001386 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2f823-f647-4e57-9607-12fad760c6d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:18.001669 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2f823-f647-4e57-9607-12fad760c6d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:18.001975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2f823-f647-4e57-9607-12fad760c6d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:18.003854 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-3cf2f823-f647-4e57-9607-12fad760c6d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:18.075665 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf2f823-f647-4e57-9607-12fad760c6d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:18.077004 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf2f823-f647-4e57-9607-12fad760c6d5 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:18.088552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1174/2372] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:13:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:13:18.821871 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e865a0-4eb9-49e0-842d-99a0cf6dc0c2 service nova] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:18.825613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-38e865a0-4eb9-49e0-842d-99a0cf6dc0c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:18.914332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e865a0-4eb9-49e0-842d-99a0cf6dc0c2 service nova] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:18.919866 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122824c-3607-46a3-9bd8-4d21409acfda service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad5cc9927374836b0cc74db85b686fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:18.920495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122824c-3607-46a3-9bd8-4d21409acfda service placement] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:18.923098 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122824c-3607-46a3-9bd8-4d21409acfda service placement] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:18.923098 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122824c-3607-46a3-9bd8-4d21409acfda service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:18.923098 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3122824c-3607-46a3-9bd8-4d21409acfda service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:18.935849 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3122824c-3607-46a3-9bd8-4d21409acfda service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:18.937501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3122824c-3607-46a3-9bd8-4d21409acfda service placement] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:18.949476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1199/2373] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:13:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:13:18.966253 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2f823-f647-4e57-9607-12fad760c6d5 service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:18.967943 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-3cf2f823-f647-4e57-9607-12fad760c6d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:18.988397 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2f823-f647-4e57-9607-12fad760c6d5 service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:18.993408 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a78ca-6032-4a68-9fdf-a1cfe921319f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:18.994261 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a78ca-6032-4a68-9fdf-a1cfe921319f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:18.994548 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a78ca-6032-4a68-9fdf-a1cfe921319f admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:18.994825 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5a78ca-6032-4a68-9fdf-a1cfe921319f admin admin] PATH_INFO: `/v3/users/f1fd384cbf4943368ee57e868c37d92c` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:18.996084 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a78ca-6032-4a68-9fdf-a1cfe921319f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1fd384cbf4943368ee57e868c37d92c)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:18.998170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5a78ca-6032-4a68-9fdf-a1cfe921319f admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:19.205604 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-fe5a78ca-6032-4a68-9fdf-a1cfe921319f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1fd384cbf4943368ee57e868c37d92c'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:19.212731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-fe5a78ca-6032-4a68-9fdf-a1cfe921319f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1fd384cbf4943368ee57e868c37d92c'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:19.213437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-fe5a78ca-6032-4a68-9fdf-a1cfe921319f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1fd384cbf4943368ee57e868c37d92c'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:19.298062 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-fe5a78ca-6032-4a68-9fdf-a1cfe921319f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1fd384cbf4943368ee57e868c37d92c'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:19.311651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-fe5a78ca-6032-4a68-9fdf-a1cfe921319f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1fd384cbf4943368ee57e868c37d92c'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:19.317837 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1175/2374] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:13:18 2022] DELETE /identity/v3/users/f1fd384cbf4943368ee57e868c37d92c => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:19.565170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122824c-3607-46a3-9bd8-4d21409acfda service placement] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:19.577271 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122824c-3607-46a3-9bd8-4d21409acfda service placement] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:19.603486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:19.636439 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d807b7-53c2-458e-a73c-7f67c6973b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:19.637081 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d807b7-53c2-458e-a73c-7f67c6973b67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:19.637354 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d807b7-53c2-458e-a73c-7f67c6973b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:19.637627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d807b7-53c2-458e-a73c-7f67c6973b67 admin admin] PATH_INFO: `/v3/projects/76cc762f121e46c096aaca327258258c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:19.638811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d807b7-53c2-458e-a73c-7f67c6973b67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76cc762f121e46c096aaca327258258c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:19.640716 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d807b7-53c2-458e-a73c-7f67c6973b67 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:19.746529 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-88d807b7-53c2-458e-a73c-7f67c6973b67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76cc762f121e46c096aaca327258258c', 'request_id': 'req-88d807b7-53c2-458e-a73c-7f67c6973b67'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:19.753049 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-88d807b7-53c2-458e-a73c-7f67c6973b67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76cc762f121e46c096aaca327258258c', 'request_id': 'req-88d807b7-53c2-458e-a73c-7f67c6973b67'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:19.846977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-88d807b7-53c2-458e-a73c-7f67c6973b67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76cc762f121e46c096aaca327258258c', 'request_id': 'req-88d807b7-53c2-458e-a73c-7f67c6973b67'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:19.859346 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-88d807b7-53c2-458e-a73c-7f67c6973b67 admin admin] The token cache is being invalidate because project 76cc762f121e46c096aaca327258258c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:13:19.859707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-88d807b7-53c2-458e-a73c-7f67c6973b67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:19.860359 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-88d807b7-53c2-458e-a73c-7f67c6973b67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:19.861999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1200/2375] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:13:19 2022] DELETE /identity/v3/projects/76cc762f121e46c096aaca327258258c => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:29.314041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:29.856303 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=39,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:31.682214 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a78ca-6032-4a68-9fdf-a1cfe921319f admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:31.683648 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-fe5a78ca-6032-4a68-9fdf-a1cfe921319f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:31.699297 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5a78ca-6032-4a68-9fdf-a1cfe921319f admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:31.756825 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ff91f-bca7-49d2-8984-b99f6ac92c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:31.757472 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ff91f-bca7-49d2-8984-b99f6ac92c7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:31.757754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ff91f-bca7-49d2-8984-b99f6ac92c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:31.758065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ff91f-bca7-49d2-8984-b99f6ac92c7b admin admin] PATH_INFO: `/v3/users/4be83520b08d4006b12b3308202447e0` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:31.759268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ff91f-bca7-49d2-8984-b99f6ac92c7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4be83520b08d4006b12b3308202447e0)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:31.761181 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41ff91f-bca7-49d2-8984-b99f6ac92c7b admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:31.926904 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e41ff91f-bca7-49d2-8984-b99f6ac92c7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4be83520b08d4006b12b3308202447e0'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:31.934571 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e41ff91f-bca7-49d2-8984-b99f6ac92c7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4be83520b08d4006b12b3308202447e0'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:31.935573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e41ff91f-bca7-49d2-8984-b99f6ac92c7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4be83520b08d4006b12b3308202447e0'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:32.008604 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e41ff91f-bca7-49d2-8984-b99f6ac92c7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4be83520b08d4006b12b3308202447e0'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:32.022126 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e41ff91f-bca7-49d2-8984-b99f6ac92c7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4be83520b08d4006b12b3308202447e0'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:32.028891 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1176/2376] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:13:31 2022] DELETE /identity/v3/users/4be83520b08d4006b12b3308202447e0 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:32.033514 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab9d25-fa42-4c64-8c5d-c59a29b4890b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:32.041173 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab9d25-fa42-4c64-8c5d-c59a29b4890b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:32.041173 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab9d25-fa42-4c64-8c5d-c59a29b4890b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:32.053825 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f5ab9d25-fa42-4c64-8c5d-c59a29b4890b None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:32.055095 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f5ab9d25-fa42-4c64-8c5d-c59a29b4890b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:32.114736 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1201/2377] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:32.124475 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ff91f-bca7-49d2-8984-b99f6ac92c7b admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:32.125849 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e41ff91f-bca7-49d2-8984-b99f6ac92c7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:32.185759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ff91f-bca7-49d2-8984-b99f6ac92c7b admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:32.196429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20f74c8-18a0-4aca-a4da-a159fb4592dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:32.197047 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f74c8-18a0-4aca-a4da-a159fb4592dd service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:32.197384 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f74c8-18a0-4aca-a4da-a159fb4592dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:32.197656 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f74c8-18a0-4aca-a4da-a159fb4592dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:32.210243 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f74c8-18a0-4aca-a4da-a159fb4592dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:32.212170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20f74c8-18a0-4aca-a4da-a159fb4592dd service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:32.222539 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1177/2378] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:13:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:13:32.639622 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab9d25-fa42-4c64-8c5d-c59a29b4890b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:32.641206 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f5ab9d25-fa42-4c64-8c5d-c59a29b4890b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:32.657810 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab9d25-fa42-4c64-8c5d-c59a29b4890b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:32.663717 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433b514-3b25-47b0-bc72-c9558f0a732d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:32.665329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433b514-3b25-47b0-bc72-c9558f0a732d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:32.666063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433b514-3b25-47b0-bc72-c9558f0a732d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:32.666428 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433b514-3b25-47b0-bc72-c9558f0a732d admin admin] PATH_INFO: `/v3/projects/5ca601aa79704545982a0a53c4bffe3d` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:32.667659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433b514-3b25-47b0-bc72-c9558f0a732d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ca601aa79704545982a0a53c4bffe3d)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:32.669673 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433b514-3b25-47b0-bc72-c9558f0a732d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:32.846409 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5433b514-3b25-47b0-bc72-c9558f0a732d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ca601aa79704545982a0a53c4bffe3d', 'request_id': 'req-5433b514-3b25-47b0-bc72-c9558f0a732d'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:32.852993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5433b514-3b25-47b0-bc72-c9558f0a732d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ca601aa79704545982a0a53c4bffe3d', 'request_id': 'req-5433b514-3b25-47b0-bc72-c9558f0a732d'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:32.942935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5433b514-3b25-47b0-bc72-c9558f0a732d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ca601aa79704545982a0a53c4bffe3d', 'request_id': 'req-5433b514-3b25-47b0-bc72-c9558f0a732d'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:32.954673 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5433b514-3b25-47b0-bc72-c9558f0a732d admin admin] The token cache is being invalidate because project 5ca601aa79704545982a0a53c4bffe3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:13:32.957238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5433b514-3b25-47b0-bc72-c9558f0a732d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:32.957238 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5433b514-3b25-47b0-bc72-c9558f0a732d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:13:32.957451 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1202/2379] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:13:32 2022] DELETE /identity/v3/projects/5ca601aa79704545982a0a53c4bffe3d => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:32.962328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672089f6-161f-4ee4-a1c0-34764d3eccd1 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:32.963436 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672089f6-161f-4ee4-a1c0-34764d3eccd1 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:32.963436 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672089f6-161f-4ee4-a1c0-34764d3eccd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:32.982631 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-672089f6-161f-4ee4-a1c0-34764d3eccd1 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:32.983830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-672089f6-161f-4ee4-a1c0-34764d3eccd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:33.002001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:33.046373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1178/2380] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:33.054296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48078e-eace-42a0-8b33-527956c4cee3 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:33.059033 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48078e-eace-42a0-8b33-527956c4cee3 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:33.059033 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff48078e-eace-42a0-8b33-527956c4cee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:33.075671 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-ff48078e-eace-42a0-8b33-527956c4cee3 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:33.076887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ff48078e-eace-42a0-8b33-527956c4cee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:33.085267 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1203/2381] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:13:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:33.089653 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672089f6-161f-4ee4-a1c0-34764d3eccd1 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:33.099910 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672089f6-161f-4ee4-a1c0-34764d3eccd1 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:33.105083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228cf274-8e6c-4d5f-91ed-99a694807f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:33.105691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228cf274-8e6c-4d5f-91ed-99a694807f47 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:33.106008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228cf274-8e6c-4d5f-91ed-99a694807f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:33.106289 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228cf274-8e6c-4d5f-91ed-99a694807f47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:33.107056 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228cf274-8e6c-4d5f-91ed-99a694807f47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:13:33.107293 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228cf274-8e6c-4d5f-91ed-99a694807f47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:33.108953 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228cf274-8e6c-4d5f-91ed-99a694807f47 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:33.121115 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1179/2382] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:13:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:13:33.125537 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48078e-eace-42a0-8b33-527956c4cee3 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:33.134921 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff48078e-eace-42a0-8b33-527956c4cee3 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:33.139678 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000a968-38b0-4940-928d-1f9f2a5b4c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:33.140284 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000a968-38b0-4940-928d-1f9f2a5b4c1e admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:33.140570 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000a968-38b0-4940-928d-1f9f2a5b4c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:33.140847 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1000a968-38b0-4940-928d-1f9f2a5b4c1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:33.141506 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1000a968-38b0-4940-928d-1f9f2a5b4c1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:33.143643 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1000a968-38b0-4940-928d-1f9f2a5b4c1e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:33.233687 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1204/2383] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:13:33 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:13:33.238201 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228cf274-8e6c-4d5f-91ed-99a694807f47 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:33.249383 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228cf274-8e6c-4d5f-91ed-99a694807f47 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:33.311296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c34301-b9c8-4e06-81dd-cf037181eb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:33.311937 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c34301-b9c8-4e06-81dd-cf037181eb97 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:33.312747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c34301-b9c8-4e06-81dd-cf037181eb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:33.312747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c34301-b9c8-4e06-81dd-cf037181eb97 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:33.313215 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c34301-b9c8-4e06-81dd-cf037181eb97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:33.315187 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c34301-b9c8-4e06-81dd-cf037181eb97 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:33.423031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1180/2384] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:13:33 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:13:33.429160 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000a968-38b0-4940-928d-1f9f2a5b4c1e admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:33.444102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1000a968-38b0-4940-928d-1f9f2a5b4c1e admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:33.448926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38105ab8-497f-489a-b01c-42b5b238157d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:33.449911 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38105ab8-497f-489a-b01c-42b5b238157d admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:33.449911 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38105ab8-497f-489a-b01c-42b5b238157d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:33.450245 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38105ab8-497f-489a-b01c-42b5b238157d admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:33.450966 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38105ab8-497f-489a-b01c-42b5b238157d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:33.452603 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38105ab8-497f-489a-b01c-42b5b238157d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:33.463763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1205/2385] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:13:33.468171 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c34301-b9c8-4e06-81dd-cf037181eb97 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:33.478892 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c34301-b9c8-4e06-81dd-cf037181eb97 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:33.483983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc8979-dd07-423e-8935-3ffc8dc004b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:33.484588 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc8979-dd07-423e-8935-3ffc8dc004b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:33.484869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc8979-dd07-423e-8935-3ffc8dc004b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:33.485144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc8979-dd07-423e-8935-3ffc8dc004b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:33.485831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc8979-dd07-423e-8935-3ffc8dc004b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:33.487455 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc8979-dd07-423e-8935-3ffc8dc004b9 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:33.498232 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1181/2386] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:13:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:13:33.502787 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38105ab8-497f-489a-b01c-42b5b238157d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:33.513443 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38105ab8-497f-489a-b01c-42b5b238157d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:33.518833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed084d-353c-4c39-b32d-4a2121202a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:33.519497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed084d-353c-4c39-b32d-4a2121202a09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:33.519785 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed084d-353c-4c39-b32d-4a2121202a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:33.520065 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ed084d-353c-4c39-b32d-4a2121202a09 admin admin] PATH_INFO: `/v3/projects/d65888d17d2e439592600a4e9305eeed/users/02569c995a6d4b678933599f60ad3909/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:33.535305 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ed084d-353c-4c39-b32d-4a2121202a09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d65888d17d2e439592600a4e9305eeed, user_id=02569c995a6d4b678933599f60ad3909, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:33.537982 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ed084d-353c-4c39-b32d-4a2121202a09 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:33.611875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1206/2387] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:13:33 2022] PUT /identity/v3/projects/d65888d17d2e439592600a4e9305eeed/users/02569c995a6d4b678933599f60ad3909/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:33.617048 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70cd57b-fb09-4b8e-aa2d-5814ad0d6e8c None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:33.617342 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70cd57b-fb09-4b8e-aa2d-5814ad0d6e8c None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:33.617621 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70cd57b-fb09-4b8e-aa2d-5814ad0d6e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:33.636718 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-b70cd57b-fb09-4b8e-aa2d-5814ad0d6e8c None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:33.637968 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-b70cd57b-fb09-4b8e-aa2d-5814ad0d6e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:33.699249 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1182/2388] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:13:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:33.709096 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed084d-353c-4c39-b32d-4a2121202a09 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:33.710857 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-10ed084d-353c-4c39-b32d-4a2121202a09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:33.735055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:33.770859 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ed084d-353c-4c39-b32d-4a2121202a09 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:33.775175 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047886f-fed5-4741-b87d-0f1b794109ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:33.775789 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047886f-fed5-4741-b87d-0f1b794109ef service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:33.776112 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047886f-fed5-4741-b87d-0f1b794109ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:33.776398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047886f-fed5-4741-b87d-0f1b794109ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:33.785608 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047886f-fed5-4741-b87d-0f1b794109ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:33.787078 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047886f-fed5-4741-b87d-0f1b794109ef service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:33.798273 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1207/2389] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:13:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:13:39.326570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d2720-f101-4000-a420-e2f1e70c5743 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:39.327321 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d2720-f101-4000-a420-e2f1e70c5743 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:39.327321 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d2720-f101-4000-a420-e2f1e70c5743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:39.359888 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-9a7d2720-f101-4000-a420-e2f1e70c5743 None None] MFA Rules not processed for user `02569c995a6d4b678933599f60ad3909`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:39.361137 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-9a7d2720-f101-4000-a420-e2f1e70c5743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:39.382694 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:39.442247 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1183/2390] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:13:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:39.446860 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047886f-fed5-4741-b87d-0f1b794109ef service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:39.457721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047886f-fed5-4741-b87d-0f1b794109ef service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:39.464656 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3cffe-6abc-46de-927d-16bec8dd04d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:39.465290 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3cffe-6abc-46de-927d-16bec8dd04d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:39.465572 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3cffe-6abc-46de-927d-16bec8dd04d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:39.465848 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3cffe-6abc-46de-927d-16bec8dd04d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:39.466560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d3cffe-6abc-46de-927d-16bec8dd04d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:39.468505 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d3cffe-6abc-46de-927d-16bec8dd04d5 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:39.594468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1208/2391] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:13:39 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:13:39.598622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d2720-f101-4000-a420-e2f1e70c5743 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:39.610704 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d2720-f101-4000-a420-e2f1e70c5743 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:39.674805 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c927e8-6327-45c5-97e0-aefdc7cce1ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:39.675543 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c927e8-6327-45c5-97e0-aefdc7cce1ab admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:39.675823 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c927e8-6327-45c5-97e0-aefdc7cce1ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:39.676090 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c927e8-6327-45c5-97e0-aefdc7cce1ab admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:39.676779 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c927e8-6327-45c5-97e0-aefdc7cce1ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:39.678775 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c927e8-6327-45c5-97e0-aefdc7cce1ab admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:39.792335 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1184/2392] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:13:39 2022] POST /identity/v3/users => generated 296 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:13:39.796407 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3cffe-6abc-46de-927d-16bec8dd04d5 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:39.807021 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3cffe-6abc-46de-927d-16bec8dd04d5 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:39.811662 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c712b-11c3-418e-980c-adf824fd644a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:39.812285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c712b-11c3-418e-980c-adf824fd644a admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:39.812571 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c712b-11c3-418e-980c-adf824fd644a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:39.812846 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c712b-11c3-418e-980c-adf824fd644a admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:39.813569 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c712b-11c3-418e-980c-adf824fd644a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:39.815638 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c712b-11c3-418e-980c-adf824fd644a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:39.826980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1209/2393] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:13:39.831347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c927e8-6327-45c5-97e0-aefdc7cce1ab admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:39.842664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c927e8-6327-45c5-97e0-aefdc7cce1ab admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:39.847544 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2a1fe-fdf3-4c43-876e-586ddbee2a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:39.848221 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2a1fe-fdf3-4c43-876e-586ddbee2a3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:39.848504 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2a1fe-fdf3-4c43-876e-586ddbee2a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:39.848905 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2a1fe-fdf3-4c43-876e-586ddbee2a3b admin admin] PATH_INFO: `/v3/projects/8fd19ae801584f7e899b3b3fb2a36136/users/5077255a76fc45d9a62557db042932da/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:39.864084 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2a1fe-fdf3-4c43-876e-586ddbee2a3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd19ae801584f7e899b3b3fb2a36136, user_id=5077255a76fc45d9a62557db042932da, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:39.866785 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2a1fe-fdf3-4c43-876e-586ddbee2a3b admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:39.969523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1185/2394] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:13:39 2022] PUT /identity/v3/projects/8fd19ae801584f7e899b3b3fb2a36136/users/5077255a76fc45d9a62557db042932da/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:39.973802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c712b-11c3-418e-980c-adf824fd644a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:39.984096 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c712b-11c3-418e-980c-adf824fd644a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:40.045445 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bdd47-58a1-432e-beaa-23403a8a222f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:40.046141 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bdd47-58a1-432e-beaa-23403a8a222f admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:40.046429 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bdd47-58a1-432e-beaa-23403a8a222f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:40.046708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bdd47-58a1-432e-beaa-23403a8a222f admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:40.047481 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7bdd47-58a1-432e-beaa-23403a8a222f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:40.049098 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7bdd47-58a1-432e-beaa-23403a8a222f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:40.060002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1210/2395] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:13:40.064284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2a1fe-fdf3-4c43-876e-586ddbee2a3b admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:40.075733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2a1fe-fdf3-4c43-876e-586ddbee2a3b admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:40.080978 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef92af8-79ee-4ffd-96ce-4a3876149b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:40.081641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef92af8-79ee-4ffd-96ce-4a3876149b51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:40.081962 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef92af8-79ee-4ffd-96ce-4a3876149b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:40.082241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef92af8-79ee-4ffd-96ce-4a3876149b51 admin admin] PATH_INFO: `/v3/projects/8fd19ae801584f7e899b3b3fb2a36136/users/5077255a76fc45d9a62557db042932da/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:40.084258 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef92af8-79ee-4ffd-96ce-4a3876149b51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd19ae801584f7e899b3b3fb2a36136, user_id=5077255a76fc45d9a62557db042932da, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:40.088780 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef92af8-79ee-4ffd-96ce-4a3876149b51 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:40.099724 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1186/2396] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:13:40 2022] PUT /identity/v3/projects/8fd19ae801584f7e899b3b3fb2a36136/users/5077255a76fc45d9a62557db042932da/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:13:46.489374 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2dae25-c031-450b-a952-3b920bb85da0 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:46.489979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2dae25-c031-450b-a952-3b920bb85da0 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:46.489979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2dae25-c031-450b-a952-3b920bb85da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:46.521154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-bb2dae25-c031-450b-a952-3b920bb85da0 None None] MFA Rules not processed for user `5077255a76fc45d9a62557db042932da`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:13:46.522425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bb2dae25-c031-450b-a952-3b920bb85da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:46.595456 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1211/2397] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:13:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:13:46.609306 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef92af8-79ee-4ffd-96ce-4a3876149b51 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:46.611331 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-3ef92af8-79ee-4ffd-96ce-4a3876149b51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:46.684371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef92af8-79ee-4ffd-96ce-4a3876149b51 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:46.689770 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562d6b3-2a83-47b8-8b6f-476950a764d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:46.691645 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562d6b3-2a83-47b8-8b6f-476950a764d4 service nova] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:46.691645 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562d6b3-2a83-47b8-8b6f-476950a764d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:46.691645 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562d6b3-2a83-47b8-8b6f-476950a764d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:46.696875 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:46.750538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562d6b3-2a83-47b8-8b6f-476950a764d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:46.752125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562d6b3-2a83-47b8-8b6f-476950a764d4 service nova] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:46.762281 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1187/2398] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:13:46 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:13:52.130396 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2dae25-c031-450b-a952-3b920bb85da0 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:52.143682 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2dae25-c031-450b-a952-3b920bb85da0 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:52.149061 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fdb4b9-7865-4b6f-a8e9-e0e038d47a3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:52.149683 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdb4b9-7865-4b6f-a8e9-e0e038d47a3e service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:52.150003 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdb4b9-7865-4b6f-a8e9-e0e038d47a3e service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:52.150282 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdb4b9-7865-4b6f-a8e9-e0e038d47a3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:52.152043 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-74fdb4b9-7865-4b6f-a8e9-e0e038d47a3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:52.161952 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:13:52.214308 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fdb4b9-7865-4b6f-a8e9-e0e038d47a3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:52.215884 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fdb4b9-7865-4b6f-a8e9-e0e038d47a3e service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:52.226154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1212/2399] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:13:52 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:13:53.032025 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562d6b3-2a83-47b8-8b6f-476950a764d4 service nova] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:13:53.033719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-5562d6b3-2a83-47b8-8b6f-476950a764d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:53.049927 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562d6b3-2a83-47b8-8b6f-476950a764d4 service nova] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:13:53.056255 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec2a30-87e6-400f-9341-f8827a1a7f15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:13:53.056822 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec2a30-87e6-400f-9341-f8827a1a7f15 service nova] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:13:53.057107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec2a30-87e6-400f-9341-f8827a1a7f15 service nova] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:13:53.057388 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec2a30-87e6-400f-9341-f8827a1a7f15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:13:53.059188 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-a5ec2a30-87e6-400f-9341-f8827a1a7f15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:13:53.072795 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec2a30-87e6-400f-9341-f8827a1a7f15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:13:53.074708 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ec2a30-87e6-400f-9341-f8827a1a7f15 service nova] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:13:53.094480 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1188/2400] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:13:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:14:02.304553 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=23 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:14:02.496999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cdd03-ebaa-44a4-8a87-3018397bdca8 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:02.497290 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cdd03-ebaa-44a4-8a87-3018397bdca8 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:02.497573 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023cdd03-ebaa-44a4-8a87-3018397bdca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:02.517255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-023cdd03-ebaa-44a4-8a87-3018397bdca8 None None] MFA Rules not processed for user `ed8114b7a28741268fdff71c866dbc77`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:14:02.518491 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-023cdd03-ebaa-44a4-8a87-3018397bdca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:02.579998 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1213/2401] 10.222.0.46 () {58 vars in 967 bytes} [Tue Aug 2 09:14:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:14:02.765671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec2a30-87e6-400f-9341-f8827a1a7f15 service nova] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:02.767307 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-a5ec2a30-87e6-400f-9341-f8827a1a7f15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:02.842022 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ec2a30-87e6-400f-9341-f8827a1a7f15 service nova] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:02.847756 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2688c-d2ad-454b-8ce1-8891adfbbd51 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff92ef3b9da74ccab3fbc43f852e3ad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:02.848315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2688c-d2ad-454b-8ce1-8891adfbbd51 service swift] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:02.848315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2688c-d2ad-454b-8ce1-8891adfbbd51 service swift] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:02.848315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb2688c-d2ad-454b-8ce1-8891adfbbd51 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:02.860926 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2688c-d2ad-454b-8ce1-8891adfbbd51 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:02.862983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb2688c-d2ad-454b-8ce1-8891adfbbd51 service swift] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:02.872557 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1189/2402] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 2 09:14:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:14:03.880672 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cdd03-ebaa-44a4-8a87-3018397bdca8 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:03.882135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-023cdd03-ebaa-44a4-8a87-3018397bdca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:03.939865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023cdd03-ebaa-44a4-8a87-3018397bdca8 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:03.943595 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0244f-26cf-421f-b0d4-47771112d83e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:03.944220 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0244f-26cf-421f-b0d4-47771112d83e service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:03.944509 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0244f-26cf-421f-b0d4-47771112d83e service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:03.944789 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0244f-26cf-421f-b0d4-47771112d83e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:03.947180 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-20b0244f-26cf-421f-b0d4-47771112d83e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:04.013163 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0244f-26cf-421f-b0d4-47771112d83e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:04.014633 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0244f-26cf-421f-b0d4-47771112d83e service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:04.025291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1214/2403] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:14:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:14:08.790372 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2688c-d2ad-454b-8ce1-8891adfbbd51 service swift] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:08.792738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-7bb2688c-d2ad-454b-8ce1-8891adfbbd51 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:08.884443 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb2688c-d2ad-454b-8ce1-8891adfbbd51 service swift] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:08.890848 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b585d2-018e-4bb8-a3d5-3cfdefdcfdf8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad5cc9927374836b0cc74db85b686fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:08.891486 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b585d2-018e-4bb8-a3d5-3cfdefdcfdf8 service placement] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:08.891772 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b585d2-018e-4bb8-a3d5-3cfdefdcfdf8 service placement] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:08.892053 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b585d2-018e-4bb8-a3d5-3cfdefdcfdf8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:08.893833 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-62b585d2-018e-4bb8-a3d5-3cfdefdcfdf8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:08.910573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b585d2-018e-4bb8-a3d5-3cfdefdcfdf8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:08.912182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b585d2-018e-4bb8-a3d5-3cfdefdcfdf8 service placement] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:08.925169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1190/2404] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:14:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:14:08.987253 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8860e0c-343f-4e0b-9b41-0003804bc50d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:08.987791 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8860e0c-343f-4e0b-9b41-0003804bc50d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:08.988094 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8860e0c-343f-4e0b-9b41-0003804bc50d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:09.024054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f8860e0c-343f-4e0b-9b41-0003804bc50d None None] MFA Rules not processed for user `7fd6b148658e4c2884a32f66326833b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:14:09.025479 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f8860e0c-343f-4e0b-9b41-0003804bc50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:09.110194 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:14:09.122067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1215/2405] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:14:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:14:13.146132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c045eb6-c796-43bf-9db5-1fcc1af8dd26 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:13.147011 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c045eb6-c796-43bf-9db5-1fcc1af8dd26 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:13.147450 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c045eb6-c796-43bf-9db5-1fcc1af8dd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:13.169700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-4c045eb6-c796-43bf-9db5-1fcc1af8dd26 None None] MFA Rules not processed for user `806f0c03435e4c5daeadaa4eb437d936`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:14:13.171088 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4c045eb6-c796-43bf-9db5-1fcc1af8dd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:13.180834 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:14:13.236512 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1191/2406] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:14:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:14:13.913226 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8860e0c-343f-4e0b-9b41-0003804bc50d None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:13.915896 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f8860e0c-343f-4e0b-9b41-0003804bc50d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:13.950388 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8860e0c-343f-4e0b-9b41-0003804bc50d None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:13.958927 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4e22c-4516-45e8-9d56-384e27a9643e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:13.959768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4e22c-4516-45e8-9d56-384e27a9643e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:13.960307 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4e22c-4516-45e8-9d56-384e27a9643e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:13.961683 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4e22c-4516-45e8-9d56-384e27a9643e admin admin] PATH_INFO: `/v3/users/8b2daf470f45422a8c1bd5cf62efdd9e` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:13.963755 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4e22c-4516-45e8-9d56-384e27a9643e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b2daf470f45422a8c1bd5cf62efdd9e)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:13.966730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4e22c-4516-45e8-9d56-384e27a9643e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:14.000074 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd3d08-4904-42f3-ac21-644c8980b053 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:14.000663 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd3d08-4904-42f3-ac21-644c8980b053 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:14.000946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd3d08-4904-42f3-ac21-644c8980b053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:14.024106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-cabd3d08-4904-42f3-ac21-644c8980b053 None None] MFA Rules not processed for user `02569c995a6d4b678933599f60ad3909`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:14:14.025510 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-cabd3d08-4904-42f3-ac21-644c8980b053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:14.032621 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1192/2407] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:14:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:14:14.183516 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fed4e22c-4516-45e8-9d56-384e27a9643e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b2daf470f45422a8c1bd5cf62efdd9e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:14:14.193267 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fed4e22c-4516-45e8-9d56-384e27a9643e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b2daf470f45422a8c1bd5cf62efdd9e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:14:14.194477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fed4e22c-4516-45e8-9d56-384e27a9643e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b2daf470f45422a8c1bd5cf62efdd9e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:14:14.209557 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fed4e22c-4516-45e8-9d56-384e27a9643e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b2daf470f45422a8c1bd5cf62efdd9e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:14:14.210780 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fed4e22c-4516-45e8-9d56-384e27a9643e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b2daf470f45422a8c1bd5cf62efdd9e'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:14:14.281011 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1216/2408] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:14:13 2022] DELETE /identity/v3/users/8b2daf470f45422a8c1bd5cf62efdd9e => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:14:14.394328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd3d08-4904-42f3-ac21-644c8980b053 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:14.407336 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd3d08-4904-42f3-ac21-644c8980b053 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:14.465893 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c30277-a664-4e5b-b787-5b2c1743fc00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:14.466533 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c30277-a664-4e5b-b787-5b2c1743fc00 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:14.466821 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c30277-a664-4e5b-b787-5b2c1743fc00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:14.467094 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c30277-a664-4e5b-b787-5b2c1743fc00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:14.468897 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-d8c30277-a664-4e5b-b787-5b2c1743fc00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:14.482770 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c30277-a664-4e5b-b787-5b2c1743fc00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:14.484485 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c30277-a664-4e5b-b787-5b2c1743fc00 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:14.494626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1193/2409] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:14:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:14:14.584640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4e22c-4516-45e8-9d56-384e27a9643e admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:14.586991 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-fed4e22c-4516-45e8-9d56-384e27a9643e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:14.727280 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4e22c-4516-45e8-9d56-384e27a9643e admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:14.732915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a739cba-47ec-41b7-949a-2a82c475fc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:14.733728 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a739cba-47ec-41b7-949a-2a82c475fc8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:14.734163 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a739cba-47ec-41b7-949a-2a82c475fc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:14.734537 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a739cba-47ec-41b7-949a-2a82c475fc8f admin admin] PATH_INFO: `/v3/projects/5132918a404e4b6ab13577698e319630` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:14.735875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a739cba-47ec-41b7-949a-2a82c475fc8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5132918a404e4b6ab13577698e319630)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:14.743373 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a739cba-47ec-41b7-949a-2a82c475fc8f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:14.877832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-2a739cba-47ec-41b7-949a-2a82c475fc8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5132918a404e4b6ab13577698e319630', 'request_id': 'req-2a739cba-47ec-41b7-949a-2a82c475fc8f'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:14:14.885281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-2a739cba-47ec-41b7-949a-2a82c475fc8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5132918a404e4b6ab13577698e319630', 'request_id': 'req-2a739cba-47ec-41b7-949a-2a82c475fc8f'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:14:14.960232 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-2a739cba-47ec-41b7-949a-2a82c475fc8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5132918a404e4b6ab13577698e319630', 'request_id': 'req-2a739cba-47ec-41b7-949a-2a82c475fc8f'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:14:14.982576 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-2a739cba-47ec-41b7-949a-2a82c475fc8f admin admin] The token cache is being invalidate because project 5132918a404e4b6ab13577698e319630 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:14:14.983211 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-2a739cba-47ec-41b7-949a-2a82c475fc8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:14:14.985568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-2a739cba-47ec-41b7-949a-2a82c475fc8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:14:14.987602 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1217/2410] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:14:14 2022] DELETE /identity/v3/projects/5132918a404e4b6ab13577698e319630 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:14:14.991812 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c30277-a664-4e5b-b787-5b2c1743fc00 service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:15.011612 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c30277-a664-4e5b-b787-5b2c1743fc00 service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:15.098547 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f857d8-1614-4a59-8cf1-90543767deb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:15.099205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f857d8-1614-4a59-8cf1-90543767deb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:15.099491 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f857d8-1614-4a59-8cf1-90543767deb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:15.099773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f857d8-1614-4a59-8cf1-90543767deb5 admin admin] PATH_INFO: `/v3/users/f2526b051be64aa0845f665d73bf9bf6` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:15.101931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f857d8-1614-4a59-8cf1-90543767deb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2526b051be64aa0845f665d73bf9bf6)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:15.104554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f857d8-1614-4a59-8cf1-90543767deb5 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:15.361445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-85f857d8-1614-4a59-8cf1-90543767deb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2526b051be64aa0845f665d73bf9bf6'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:14:15.372103 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-85f857d8-1614-4a59-8cf1-90543767deb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2526b051be64aa0845f665d73bf9bf6'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:14:15.372965 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-85f857d8-1614-4a59-8cf1-90543767deb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2526b051be64aa0845f665d73bf9bf6'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:14:15.375788 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:14:15.518260 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-85f857d8-1614-4a59-8cf1-90543767deb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2526b051be64aa0845f665d73bf9bf6'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:14:15.542795 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-85f857d8-1614-4a59-8cf1-90543767deb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2526b051be64aa0845f665d73bf9bf6'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:14:15.562287 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1194/2411] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:14:14 2022] DELETE /identity/v3/users/f2526b051be64aa0845f665d73bf9bf6 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:14:15.880577 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a739cba-47ec-41b7-949a-2a82c475fc8f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:15.882147 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2a739cba-47ec-41b7-949a-2a82c475fc8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:15.890900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:14:15.897847 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a739cba-47ec-41b7-949a-2a82c475fc8f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:15.962302 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32c9ca5-1ca1-4aa3-9632-fb97db8e3b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:15.963700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c9ca5-1ca1-4aa3-9632-fb97db8e3b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:15.963975 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c9ca5-1ca1-4aa3-9632-fb97db8e3b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:15.964258 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32c9ca5-1ca1-4aa3-9632-fb97db8e3b6c admin admin] PATH_INFO: `/v3/projects/9e6f2eeeb8d84656824effb901fc9499` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:15.965870 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c9ca5-1ca1-4aa3-9632-fb97db8e3b6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e6f2eeeb8d84656824effb901fc9499)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:15.968438 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32c9ca5-1ca1-4aa3-9632-fb97db8e3b6c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:16.103922 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c32c9ca5-1ca1-4aa3-9632-fb97db8e3b6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e6f2eeeb8d84656824effb901fc9499', 'request_id': 'req-c32c9ca5-1ca1-4aa3-9632-fb97db8e3b6c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:14:16.113390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c32c9ca5-1ca1-4aa3-9632-fb97db8e3b6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e6f2eeeb8d84656824effb901fc9499', 'request_id': 'req-c32c9ca5-1ca1-4aa3-9632-fb97db8e3b6c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:14:16.171570 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c32c9ca5-1ca1-4aa3-9632-fb97db8e3b6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e6f2eeeb8d84656824effb901fc9499', 'request_id': 'req-c32c9ca5-1ca1-4aa3-9632-fb97db8e3b6c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:14:16.183723 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c32c9ca5-1ca1-4aa3-9632-fb97db8e3b6c admin admin] The token cache is being invalidate because project 9e6f2eeeb8d84656824effb901fc9499 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:14:16.184017 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c32c9ca5-1ca1-4aa3-9632-fb97db8e3b6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:14:16.184837 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c32c9ca5-1ca1-4aa3-9632-fb97db8e3b6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:14:16.187404 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1218/2412] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:14:15 2022] DELETE /identity/v3/projects/9e6f2eeeb8d84656824effb901fc9499 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:14:16.192728 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6745418-236d-41bb-b916-98801cca9d94 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:16.193018 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6745418-236d-41bb-b916-98801cca9d94 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:16.193392 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6745418-236d-41bb-b916-98801cca9d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:16.214930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-d6745418-236d-41bb-b916-98801cca9d94 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:14:16.216349 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-d6745418-236d-41bb-b916-98801cca9d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:16.280894 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1195/2413] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:14:16.289515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503b462-6f66-4f76-b6dd-6a87366ae062 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:16.289931 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503b462-6f66-4f76-b6dd-6a87366ae062 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:16.290217 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c503b462-6f66-4f76-b6dd-6a87366ae062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:16.311569 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-c503b462-6f66-4f76-b6dd-6a87366ae062 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:14:16.312786 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-c503b462-6f66-4f76-b6dd-6a87366ae062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:16.322352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1219/2414] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:14:16.326859 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6745418-236d-41bb-b916-98801cca9d94 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:16.336504 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6745418-236d-41bb-b916-98801cca9d94 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:16.341790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1526f36-7599-439b-b0f7-d2b0cbe5c131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:16.342592 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1526f36-7599-439b-b0f7-d2b0cbe5c131 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:16.347845 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1526f36-7599-439b-b0f7-d2b0cbe5c131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:16.347845 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1526f36-7599-439b-b0f7-d2b0cbe5c131 admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:16.347845 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1526f36-7599-439b-b0f7-d2b0cbe5c131 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:14:16.347845 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1526f36-7599-439b-b0f7-d2b0cbe5c131 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:16.347845 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1526f36-7599-439b-b0f7-d2b0cbe5c131 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:16.358657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1196/2415] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:14:16.363097 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503b462-6f66-4f76-b6dd-6a87366ae062 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:16.372962 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c503b462-6f66-4f76-b6dd-6a87366ae062 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:16.377687 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6ee261-eec0-4d5f-91b3-b4eae54e5ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:16.378338 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ee261-eec0-4d5f-91b3-b4eae54e5ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:16.378623 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ee261-eec0-4d5f-91b3-b4eae54e5ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:16.378896 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6ee261-eec0-4d5f-91b3-b4eae54e5ab3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:16.379565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6ee261-eec0-4d5f-91b3-b4eae54e5ab3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:16.381463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6ee261-eec0-4d5f-91b3-b4eae54e5ab3 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:16.483159 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1220/2416] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:14:16 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:14:16.485021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1526f36-7599-439b-b0f7-d2b0cbe5c131 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:16.497514 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1526f36-7599-439b-b0f7-d2b0cbe5c131 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:16.583722 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a252440-6ea9-4403-a1b7-42aa536d68df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:16.584339 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a252440-6ea9-4403-a1b7-42aa536d68df admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:16.584783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a252440-6ea9-4403-a1b7-42aa536d68df admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:16.585062 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a252440-6ea9-4403-a1b7-42aa536d68df admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:16.587393 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a252440-6ea9-4403-a1b7-42aa536d68df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:16.588615 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a252440-6ea9-4403-a1b7-42aa536d68df admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:16.681815 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1197/2417] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:14:16 2022] POST /identity/v3/users => generated 287 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:14:16.685795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6ee261-eec0-4d5f-91b3-b4eae54e5ab3 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:16.698768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6ee261-eec0-4d5f-91b3-b4eae54e5ab3 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:16.703677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6daeb-e5a3-4080-a376-1b38d881c5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:16.704312 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6daeb-e5a3-4080-a376-1b38d881c5ec admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:16.704585 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6daeb-e5a3-4080-a376-1b38d881c5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:16.707021 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6daeb-e5a3-4080-a376-1b38d881c5ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:16.707021 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6daeb-e5a3-4080-a376-1b38d881c5ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:16.708404 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6daeb-e5a3-4080-a376-1b38d881c5ec admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:16.720931 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1221/2418] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:14:16.725916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a252440-6ea9-4403-a1b7-42aa536d68df admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:16.744119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a252440-6ea9-4403-a1b7-42aa536d68df admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:16.754559 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3a980c-d3e6-45c1-b2f6-786e8e72bdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:16.762421 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a980c-d3e6-45c1-b2f6-786e8e72bdf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:16.767758 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a980c-d3e6-45c1-b2f6-786e8e72bdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:16.767758 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3a980c-d3e6-45c1-b2f6-786e8e72bdf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:16.767758 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3a980c-d3e6-45c1-b2f6-786e8e72bdf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:16.767758 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3a980c-d3e6-45c1-b2f6-786e8e72bdf9 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:16.792305 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6daeb-e5a3-4080-a376-1b38d881c5ec admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:16.805048 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1198/2419] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:14:16.811324 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6daeb-e5a3-4080-a376-1b38d881c5ec admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:16.817834 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6709e952-ec20-4922-a0df-4269eb1bf5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:16.819020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709e952-ec20-4922-a0df-4269eb1bf5e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:16.819783 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709e952-ec20-4922-a0df-4269eb1bf5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:16.820245 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709e952-ec20-4922-a0df-4269eb1bf5e7 admin admin] PATH_INFO: `/v3/projects/142acfd7d2a7483083b9807280598eef/users/a1300b00c7214d168e72850a8585bd47/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:16.845341 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6709e952-ec20-4922-a0df-4269eb1bf5e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=142acfd7d2a7483083b9807280598eef, user_id=a1300b00c7214d168e72850a8585bd47, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:16.852577 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6709e952-ec20-4922-a0df-4269eb1bf5e7 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:16.953857 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1222/2420] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:14:16 2022] PUT /identity/v3/projects/142acfd7d2a7483083b9807280598eef/users/a1300b00c7214d168e72850a8585bd47/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:14:16.959342 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084f006-b3d9-4b32-a9e2-c057f616b95c None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:16.959787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084f006-b3d9-4b32-a9e2-c057f616b95c None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:16.960068 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b084f006-b3d9-4b32-a9e2-c057f616b95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:16.988268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-b084f006-b3d9-4b32-a9e2-c057f616b95c None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:14:16.989679 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-b084f006-b3d9-4b32-a9e2-c057f616b95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:17.029464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:14:17.060695 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1199/2421] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:14:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:14:17.078793 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6709e952-ec20-4922-a0df-4269eb1bf5e7 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:17.080108 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6709e952-ec20-4922-a0df-4269eb1bf5e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:17.116070 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:14:17.144928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6709e952-ec20-4922-a0df-4269eb1bf5e7 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:17.149700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050c6a9-0c28-4929-934f-d37e64c1c623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:17.150572 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050c6a9-0c28-4929-934f-d37e64c1c623 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:17.150835 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050c6a9-0c28-4929-934f-d37e64c1c623 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:17.151111 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050c6a9-0c28-4929-934f-d37e64c1c623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:17.161021 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050c6a9-0c28-4929-934f-d37e64c1c623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:17.162540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050c6a9-0c28-4929-934f-d37e64c1c623 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:17.173025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1223/2422] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:14:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:14:26.626708 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1242a5b-e70a-4db3-9fb2-b4bd570b0129 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:26.627208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1242a5b-e70a-4db3-9fb2-b4bd570b0129 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:26.627302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1242a5b-e70a-4db3-9fb2-b4bd570b0129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:26.633981 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1f60a-55c3-4fb6-a2ae-2422822beaf1 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:26.634866 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1f60a-55c3-4fb6-a2ae-2422822beaf1 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:26.635212 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1f60a-55c3-4fb6-a2ae-2422822beaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:26.657440 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-03e1f60a-55c3-4fb6-a2ae-2422822beaf1 None None] MFA Rules not processed for user `50918811a90a444eaf123b2aeb4a75b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:14:26.658720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-03e1f60a-55c3-4fb6-a2ae-2422822beaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:26.664457 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-b1242a5b-e70a-4db3-9fb2-b4bd570b0129 None None] MFA Rules not processed for user `a1300b00c7214d168e72850a8585bd47`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:14:26.665660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-b1242a5b-e70a-4db3-9fb2-b4bd570b0129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:26.718069 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1224/2423] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:14:26 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:14:26.739508 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1200/2424] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:14:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:14:26.744058 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1f60a-55c3-4fb6-a2ae-2422822beaf1 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:26.755768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1f60a-55c3-4fb6-a2ae-2422822beaf1 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:26.760329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c2a19-e0a3-4899-8924-b4df86a41657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:26.760976 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c2a19-e0a3-4899-8924-b4df86a41657 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:26.761266 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c2a19-e0a3-4899-8924-b4df86a41657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:26.761608 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c2a19-e0a3-4899-8924-b4df86a41657 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:26.762260 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c2a19-e0a3-4899-8924-b4df86a41657 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:26.764208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c2a19-e0a3-4899-8924-b4df86a41657 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:26.864952 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1225/2425] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:14:26 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:14:26.868983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1242a5b-e70a-4db3-9fb2-b4bd570b0129 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:26.883610 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1242a5b-e70a-4db3-9fb2-b4bd570b0129 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:26.964257 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867490b4-900b-48d7-b9d2-0e8660e3e28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:26.965050 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867490b4-900b-48d7-b9d2-0e8660e3e28e admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:26.965801 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867490b4-900b-48d7-b9d2-0e8660e3e28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:26.966142 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867490b4-900b-48d7-b9d2-0e8660e3e28e admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:26.967353 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867490b4-900b-48d7-b9d2-0e8660e3e28e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:26.969875 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867490b4-900b-48d7-b9d2-0e8660e3e28e admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:27.058682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1201/2426] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:14:26 2022] POST /identity/v3/users => generated 292 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:14:27.068711 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c2a19-e0a3-4899-8924-b4df86a41657 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:27.087018 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c2a19-e0a3-4899-8924-b4df86a41657 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:27.103336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa775d-f981-4056-8a17-43b285512655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:27.104018 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa775d-f981-4056-8a17-43b285512655 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:27.104300 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa775d-f981-4056-8a17-43b285512655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:27.104579 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fa775d-f981-4056-8a17-43b285512655 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:27.105295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa775d-f981-4056-8a17-43b285512655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:27.107189 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fa775d-f981-4056-8a17-43b285512655 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:27.133265 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1226/2427] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:14:27.133514 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867490b4-900b-48d7-b9d2-0e8660e3e28e admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:27.145408 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867490b4-900b-48d7-b9d2-0e8660e3e28e admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:27.152213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8737aa-c199-42da-91f1-bedbdf06e99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:27.154108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8737aa-c199-42da-91f1-bedbdf06e99b admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:27.154108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8737aa-c199-42da-91f1-bedbdf06e99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:27.154108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8737aa-c199-42da-91f1-bedbdf06e99b admin admin] PATH_INFO: `/v3/projects/1cb9ed01f9f74a1c90b1ab967e9a9607/users/8bf4fb63bfc34115a69917d23d68c659/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:27.174949 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8737aa-c199-42da-91f1-bedbdf06e99b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cb9ed01f9f74a1c90b1ab967e9a9607, user_id=8bf4fb63bfc34115a69917d23d68c659, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:27.178079 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8737aa-c199-42da-91f1-bedbdf06e99b admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:27.259224 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1202/2428] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:14:27 2022] PUT /identity/v3/projects/1cb9ed01f9f74a1c90b1ab967e9a9607/users/8bf4fb63bfc34115a69917d23d68c659/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:14:27.263441 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa775d-f981-4056-8a17-43b285512655 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:27.278510 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fa775d-f981-4056-8a17-43b285512655 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:27.359947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a191b0-6483-432c-a6e9-1053ea498f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:27.359947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a191b0-6483-432c-a6e9-1053ea498f5e admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:27.359947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a191b0-6483-432c-a6e9-1053ea498f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:27.359947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a191b0-6483-432c-a6e9-1053ea498f5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:27.360328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a191b0-6483-432c-a6e9-1053ea498f5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:27.360328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a191b0-6483-432c-a6e9-1053ea498f5e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:27.372869 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1227/2429] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:14:27.377241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8737aa-c199-42da-91f1-bedbdf06e99b admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:27.400925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8737aa-c199-42da-91f1-bedbdf06e99b admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:27.406888 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea579f6b-9e84-4fbd-bd5d-201d03737269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:27.411235 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea579f6b-9e84-4fbd-bd5d-201d03737269 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:27.411235 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea579f6b-9e84-4fbd-bd5d-201d03737269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:27.411235 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea579f6b-9e84-4fbd-bd5d-201d03737269 admin admin] PATH_INFO: `/v3/projects/1cb9ed01f9f74a1c90b1ab967e9a9607/users/8bf4fb63bfc34115a69917d23d68c659/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:27.411235 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea579f6b-9e84-4fbd-bd5d-201d03737269 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cb9ed01f9f74a1c90b1ab967e9a9607, user_id=8bf4fb63bfc34115a69917d23d68c659, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:27.413776 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea579f6b-9e84-4fbd-bd5d-201d03737269 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:27.430294 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1203/2430] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:14:27 2022] PUT /identity/v3/projects/1cb9ed01f9f74a1c90b1ab967e9a9607/users/8bf4fb63bfc34115a69917d23d68c659/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:14:34.860610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c016a-9e50-4f99-b631-a91a5227c87d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:34.861462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c016a-9e50-4f99-b631-a91a5227c87d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:34.861751 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c016a-9e50-4f99-b631-a91a5227c87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:34.891551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-967c016a-9e50-4f99-b631-a91a5227c87d None None] MFA Rules not processed for user `8bf4fb63bfc34115a69917d23d68c659`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:14:34.892807 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-967c016a-9e50-4f99-b631-a91a5227c87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:34.930338 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:14:34.986033 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1228/2431] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:14:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:14:34.999172 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea579f6b-9e84-4fbd-bd5d-201d03737269 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:35.001493 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-ea579f6b-9e84-4fbd-bd5d-201d03737269 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:35.035180 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:14:35.134981 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea579f6b-9e84-4fbd-bd5d-201d03737269 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:35.141209 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0fbb9-c214-42aa-9fa7-b661dd4b0502 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:35.142030 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0fbb9-c214-42aa-9fa7-b661dd4b0502 service nova] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:35.142322 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0fbb9-c214-42aa-9fa7-b661dd4b0502 service nova] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:35.142599 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0fbb9-c214-42aa-9fa7-b661dd4b0502 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:35.233661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a0fbb9-c214-42aa-9fa7-b661dd4b0502 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:35.235599 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a0fbb9-c214-42aa-9fa7-b661dd4b0502 service nova] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:35.247523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1204/2432] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:14:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:14:38.342336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63dc387-3b47-4599-85a6-735d7e846d2d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:38.342336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63dc387-3b47-4599-85a6-735d7e846d2d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:38.342336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63dc387-3b47-4599-85a6-735d7e846d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:38.364651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f63dc387-3b47-4599-85a6-735d7e846d2d None None] MFA Rules not processed for user `a1300b00c7214d168e72850a8585bd47`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:14:38.366596 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f63dc387-3b47-4599-85a6-735d7e846d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:38.381512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1229/2433] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:14:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:14:41.230636 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0fbb9-c214-42aa-9fa7-b661dd4b0502 service nova] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:14:41.232181 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f7a0fbb9-c214-42aa-9fa7-b661dd4b0502 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:41.293962 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0fbb9-c214-42aa-9fa7-b661dd4b0502 service nova] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:14:41.297586 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73606f66-594b-4910-9390-d81e978aa911 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:14:41.298260 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73606f66-594b-4910-9390-d81e978aa911 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:14:41.298612 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73606f66-594b-4910-9390-d81e978aa911 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:14:41.298957 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73606f66-594b-4910-9390-d81e978aa911 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:14:41.301009 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-73606f66-594b-4910-9390-d81e978aa911 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:14:41.353712 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:14:41.370988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73606f66-594b-4910-9390-d81e978aa911 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:14:41.372477 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73606f66-594b-4910-9390-d81e978aa911 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:14:41.382895 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1205/2434] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:14:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:14:48.356249 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=19 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:14:51.398768 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=8 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:15:03.064524 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63dc387-3b47-4599-85a6-735d7e846d2d None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:15:03.066508 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f63dc387-3b47-4599-85a6-735d7e846d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:15:03.162659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63dc387-3b47-4599-85a6-735d7e846d2d None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:15:03.170595 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198e347-0c9a-4911-a037-e13321b39fad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad5cc9927374836b0cc74db85b686fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:15:03.172120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198e347-0c9a-4911-a037-e13321b39fad service placement] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:15:03.172120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198e347-0c9a-4911-a037-e13321b39fad service placement] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:15:03.172120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198e347-0c9a-4911-a037-e13321b39fad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:15:03.173641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2198e347-0c9a-4911-a037-e13321b39fad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:15:03.187225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2198e347-0c9a-4911-a037-e13321b39fad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:15:03.189489 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2198e347-0c9a-4911-a037-e13321b39fad service placement] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:15:03.199350 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1230/2435] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:15:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:15:13.194935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=41 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:15:23.115171 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73606f66-594b-4910-9390-d81e978aa911 service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:15:23.123146 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73606f66-594b-4910-9390-d81e978aa911 service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:15:23.127047 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7680da-20b6-4730-92aa-a65acd3d33fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:15:23.127692 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7680da-20b6-4730-92aa-a65acd3d33fa service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:15:23.128011 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7680da-20b6-4730-92aa-a65acd3d33fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:15:23.128292 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7680da-20b6-4730-92aa-a65acd3d33fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:15:23.138032 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7680da-20b6-4730-92aa-a65acd3d33fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:15:23.139449 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7680da-20b6-4730-92aa-a65acd3d33fa service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:15:23.149469 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1206/2436] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:15:23 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:15:33.144582 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=6 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:15:40.480139 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198e347-0c9a-4911-a037-e13321b39fad service placement] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:15:40.490329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198e347-0c9a-4911-a037-e13321b39fad service placement] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:15:40.494138 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cccbef0-1db7-4a31-8dee-78b89f32d23c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:15:40.494715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cccbef0-1db7-4a31-8dee-78b89f32d23c service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:15:40.495001 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cccbef0-1db7-4a31-8dee-78b89f32d23c service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:15:40.495281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cccbef0-1db7-4a31-8dee-78b89f32d23c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:15:40.497041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6cccbef0-1db7-4a31-8dee-78b89f32d23c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:15:40.560778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cccbef0-1db7-4a31-8dee-78b89f32d23c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:15:40.562519 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cccbef0-1db7-4a31-8dee-78b89f32d23c service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:15:40.572707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1231/2437] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:15:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:15:43.933820 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7680da-20b6-4730-92aa-a65acd3d33fa service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:15:43.935403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-ef7680da-20b6-4730-92aa-a65acd3d33fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:15:43.952141 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7680da-20b6-4730-92aa-a65acd3d33fa service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:15:43.956903 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8bacd5-c522-4c75-95c6-f8e59686d8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:15:43.957542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8bacd5-c522-4c75-95c6-f8e59686d8cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:15:43.959430 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8bacd5-c522-4c75-95c6-f8e59686d8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:15:43.959430 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8bacd5-c522-4c75-95c6-f8e59686d8cb admin admin] PATH_INFO: `/v3/users/7fd6b148658e4c2884a32f66326833b6` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:15:43.959430 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8bacd5-c522-4c75-95c6-f8e59686d8cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fd6b148658e4c2884a32f66326833b6)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:15:43.961819 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8bacd5-c522-4c75-95c6-f8e59686d8cb admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:15:44.146934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8e8bacd5-c522-4c75-95c6-f8e59686d8cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fd6b148658e4c2884a32f66326833b6'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:15:44.154987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8e8bacd5-c522-4c75-95c6-f8e59686d8cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fd6b148658e4c2884a32f66326833b6'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:15:44.155664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8e8bacd5-c522-4c75-95c6-f8e59686d8cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fd6b148658e4c2884a32f66326833b6'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:15:44.212918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8e8bacd5-c522-4c75-95c6-f8e59686d8cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fd6b148658e4c2884a32f66326833b6'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:15:44.225300 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8e8bacd5-c522-4c75-95c6-f8e59686d8cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fd6b148658e4c2884a32f66326833b6'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:15:44.231616 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1207/2438] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:15:43 2022] DELETE /identity/v3/users/7fd6b148658e4c2884a32f66326833b6 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:15:44.481155 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cccbef0-1db7-4a31-8dee-78b89f32d23c service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:15:44.491594 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cccbef0-1db7-4a31-8dee-78b89f32d23c service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:15:44.550381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f2ab52-2fc5-4439-b0bf-e7e8ee99eda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:15:44.551029 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2ab52-2fc5-4439-b0bf-e7e8ee99eda2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:15:44.551308 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2ab52-2fc5-4439-b0bf-e7e8ee99eda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:15:44.551593 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f2ab52-2fc5-4439-b0bf-e7e8ee99eda2 admin admin] PATH_INFO: `/v3/projects/259e864bb08645c79dfab23aa56d8822` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:15:44.552748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f2ab52-2fc5-4439-b0bf-e7e8ee99eda2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=259e864bb08645c79dfab23aa56d8822)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:15:44.555212 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f2ab52-2fc5-4439-b0bf-e7e8ee99eda2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:15:44.659109 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-85f2ab52-2fc5-4439-b0bf-e7e8ee99eda2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '259e864bb08645c79dfab23aa56d8822', 'request_id': 'req-85f2ab52-2fc5-4439-b0bf-e7e8ee99eda2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:15:44.668896 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-85f2ab52-2fc5-4439-b0bf-e7e8ee99eda2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '259e864bb08645c79dfab23aa56d8822', 'request_id': 'req-85f2ab52-2fc5-4439-b0bf-e7e8ee99eda2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:15:44.755661 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-85f2ab52-2fc5-4439-b0bf-e7e8ee99eda2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '259e864bb08645c79dfab23aa56d8822', 'request_id': 'req-85f2ab52-2fc5-4439-b0bf-e7e8ee99eda2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:15:44.768392 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-85f2ab52-2fc5-4439-b0bf-e7e8ee99eda2 admin admin] The token cache is being invalidate because project 259e864bb08645c79dfab23aa56d8822 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:15:44.768685 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-85f2ab52-2fc5-4439-b0bf-e7e8ee99eda2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:15:44.769335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-85f2ab52-2fc5-4439-b0bf-e7e8ee99eda2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:15:44.770980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1232/2439] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:15:44 2022] DELETE /identity/v3/projects/259e864bb08645c79dfab23aa56d8822 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:15:44.777327 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226d6da-a6d3-4010-a232-0fcb33cc6c2c None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:15:44.777622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226d6da-a6d3-4010-a232-0fcb33cc6c2c None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:15:44.777927 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5226d6da-a6d3-4010-a232-0fcb33cc6c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:15:44.798338 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-5226d6da-a6d3-4010-a232-0fcb33cc6c2c None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:15:44.799553 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-5226d6da-a6d3-4010-a232-0fcb33cc6c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:15:44.858175 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1208/2440] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:15:44.866987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d6011-7002-47a9-9d3a-08a887117e55 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:15:44.867300 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d6011-7002-47a9-9d3a-08a887117e55 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:15:44.867581 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d6011-7002-47a9-9d3a-08a887117e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:15:44.887954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-5a8d6011-7002-47a9-9d3a-08a887117e55 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:15:44.889198 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5a8d6011-7002-47a9-9d3a-08a887117e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:15:44.897331 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1233/2441] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:15:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:15:44.902187 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226d6da-a6d3-4010-a232-0fcb33cc6c2c None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:15:44.911829 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5226d6da-a6d3-4010-a232-0fcb33cc6c2c None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:15:44.917121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0624c0f-5c17-4b00-9099-3e6e4a5a8273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:15:44.917755 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0624c0f-5c17-4b00-9099-3e6e4a5a8273 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:15:44.918073 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0624c0f-5c17-4b00-9099-3e6e4a5a8273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:15:44.918354 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0624c0f-5c17-4b00-9099-3e6e4a5a8273 admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:15:44.919086 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0624c0f-5c17-4b00-9099-3e6e4a5a8273 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:15:44.919380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0624c0f-5c17-4b00-9099-3e6e4a5a8273 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:15:44.920997 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0624c0f-5c17-4b00-9099-3e6e4a5a8273 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:15:44.933510 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1209/2442] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:15:44.938390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d6011-7002-47a9-9d3a-08a887117e55 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:15:44.950506 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d6011-7002-47a9-9d3a-08a887117e55 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:15:44.955291 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6cb7f9-ebbc-41a5-8636-8dae952c82db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:15:44.955909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cb7f9-ebbc-41a5-8636-8dae952c82db admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:15:44.956191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cb7f9-ebbc-41a5-8636-8dae952c82db admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:15:44.956471 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cb7f9-ebbc-41a5-8636-8dae952c82db admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:15:44.957151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6cb7f9-ebbc-41a5-8636-8dae952c82db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:15:44.959091 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6cb7f9-ebbc-41a5-8636-8dae952c82db admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:15:44.976154 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:15:45.078572 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1234/2443] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:15:44 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:15:45.081821 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0624c0f-5c17-4b00-9099-3e6e4a5a8273 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:15:45.093066 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0624c0f-5c17-4b00-9099-3e6e4a5a8273 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:15:45.152216 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ead1b8-3bc1-4d17-847d-d7c6b9d476a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:15:45.152836 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ead1b8-3bc1-4d17-847d-d7c6b9d476a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:15:45.153122 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ead1b8-3bc1-4d17-847d-d7c6b9d476a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:15:45.153407 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ead1b8-3bc1-4d17-847d-d7c6b9d476a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:15:45.154099 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ead1b8-3bc1-4d17-847d-d7c6b9d476a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:15:45.156025 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ead1b8-3bc1-4d17-847d-d7c6b9d476a4 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:15:45.163924 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:15:45.230080 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1210/2444] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:15:45 2022] POST /identity/v3/users => generated 288 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:15:45.234263 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6cb7f9-ebbc-41a5-8636-8dae952c82db admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:15:45.245071 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6cb7f9-ebbc-41a5-8636-8dae952c82db admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:15:45.249707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5cb4f1-2e02-4dbf-a3db-da12f018ced8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:15:45.250346 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5cb4f1-2e02-4dbf-a3db-da12f018ced8 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:15:45.250624 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5cb4f1-2e02-4dbf-a3db-da12f018ced8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:15:45.250895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5cb4f1-2e02-4dbf-a3db-da12f018ced8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:15:45.251587 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5cb4f1-2e02-4dbf-a3db-da12f018ced8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:15:45.253162 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5cb4f1-2e02-4dbf-a3db-da12f018ced8 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:15:45.263750 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1235/2445] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:15:45.267709 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ead1b8-3bc1-4d17-847d-d7c6b9d476a4 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:15:45.277817 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ead1b8-3bc1-4d17-847d-d7c6b9d476a4 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:15:45.282476 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20926c6c-eff1-4a8c-a36a-863339361d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:15:45.283028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20926c6c-eff1-4a8c-a36a-863339361d40 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:15:45.283310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20926c6c-eff1-4a8c-a36a-863339361d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:15:45.283586 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20926c6c-eff1-4a8c-a36a-863339361d40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:15:45.284273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20926c6c-eff1-4a8c-a36a-863339361d40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:15:45.285859 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20926c6c-eff1-4a8c-a36a-863339361d40 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:15:45.295721 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1211/2446] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:15:45.299851 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5cb4f1-2e02-4dbf-a3db-da12f018ced8 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:15:45.310989 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5cb4f1-2e02-4dbf-a3db-da12f018ced8 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:15:45.316014 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2486b8-c3e3-40c4-9124-88f37c39377e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:15:45.316818 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2486b8-c3e3-40c4-9124-88f37c39377e admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:15:45.322391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2486b8-c3e3-40c4-9124-88f37c39377e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:15:45.322391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2486b8-c3e3-40c4-9124-88f37c39377e admin admin] PATH_INFO: `/v3/projects/bfaca149a97a44cf84f9d21fa65510b6/users/77aff587ea964860895cd4dce03f2158/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:15:45.332373 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2486b8-c3e3-40c4-9124-88f37c39377e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfaca149a97a44cf84f9d21fa65510b6, user_id=77aff587ea964860895cd4dce03f2158, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:15:45.335124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2486b8-c3e3-40c4-9124-88f37c39377e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:15:45.406912 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1236/2447] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:15:45 2022] PUT /identity/v3/projects/bfaca149a97a44cf84f9d21fa65510b6/users/77aff587ea964860895cd4dce03f2158/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:15:45.412066 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db5a84-ee08-425b-b69b-81cfe99ec42d None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:15:45.412353 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db5a84-ee08-425b-b69b-81cfe99ec42d None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:15:45.412633 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db5a84-ee08-425b-b69b-81cfe99ec42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:15:45.431831 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-d0db5a84-ee08-425b-b69b-81cfe99ec42d None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:15:45.433968 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-d0db5a84-ee08-425b-b69b-81cfe99ec42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:15:45.495554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1212/2448] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:15:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:15:45.505401 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2486b8-c3e3-40c4-9124-88f37c39377e admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:15:45.507177 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-cc2486b8-c3e3-40c4-9124-88f37c39377e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:15:45.565126 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2486b8-c3e3-40c4-9124-88f37c39377e admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:15:45.568691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0fdbcf-5196-4b40-af18-56cb4861d392 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:15:45.569310 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0fdbcf-5196-4b40-af18-56cb4861d392 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:15:45.569600 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0fdbcf-5196-4b40-af18-56cb4861d392 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:15:45.569878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0fdbcf-5196-4b40-af18-56cb4861d392 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:15:45.579563 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0fdbcf-5196-4b40-af18-56cb4861d392 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:15:45.581009 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0fdbcf-5196-4b40-af18-56cb4861d392 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:15:45.590856 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1237/2449] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:15:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:15:51.053991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1207f-4c40-4725-b9e1-c252efd70535 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:15:51.055142 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1207f-4c40-4725-b9e1-c252efd70535 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:15:51.055394 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac1207f-4c40-4725-b9e1-c252efd70535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:15:51.088343 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-9ac1207f-4c40-4725-b9e1-c252efd70535 None None] MFA Rules not processed for user `77aff587ea964860895cd4dce03f2158`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:15:51.089576 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-9ac1207f-4c40-4725-b9e1-c252efd70535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:15:51.159943 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1213/2450] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:15:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:15:51.176902 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0fdbcf-5196-4b40-af18-56cb4861d392 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:15:51.178533 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6e0fdbcf-5196-4b40-af18-56cb4861d392 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:15:51.277313 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0fdbcf-5196-4b40-af18-56cb4861d392 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:15:51.290445 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd686f-4ef6-4cbf-9007-a794e68fcfc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:15:51.290445 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd686f-4ef6-4cbf-9007-a794e68fcfc2 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:15:51.290445 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd686f-4ef6-4cbf-9007-a794e68fcfc2 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:15:51.290445 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfd686f-4ef6-4cbf-9007-a794e68fcfc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:15:51.295554 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd686f-4ef6-4cbf-9007-a794e68fcfc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:15:51.297322 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfd686f-4ef6-4cbf-9007-a794e68fcfc2 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:15:51.315286 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1238/2451] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:15:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:15:53.659479 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0bc06-4bdd-47b7-896b-927ada3b6c65 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:15:53.659946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0bc06-4bdd-47b7-896b-927ada3b6c65 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:15:53.660042 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0bc06-4bdd-47b7-896b-927ada3b6c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:15:53.678598 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-6ff0bc06-4bdd-47b7-896b-927ada3b6c65 None None] MFA Rules not processed for user `77aff587ea964860895cd4dce03f2158`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:15:53.680366 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-6ff0bc06-4bdd-47b7-896b-927ada3b6c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:15:53.687600 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1214/2452] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:15:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:16:01.303499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=77,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:16:03.673934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:INSERT=3,keystone:SELECT=69 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:16:16.838639 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd686f-4ef6-4cbf-9007-a794e68fcfc2 service nova] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:16:16.840153 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3cfd686f-4ef6-4cbf-9007-a794e68fcfc2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:16:16.898534 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfd686f-4ef6-4cbf-9007-a794e68fcfc2 service nova] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:16:16.901873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7c23b6-72a4-477f-b585-19c386d88cc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:16:16.902550 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c23b6-72a4-477f-b585-19c386d88cc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:16:16.902840 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c23b6-72a4-477f-b585-19c386d88cc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:16:16.903120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7c23b6-72a4-477f-b585-19c386d88cc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:16:16.904961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7b7c23b6-72a4-477f-b585-19c386d88cc8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:16:16.918854 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c23b6-72a4-477f-b585-19c386d88cc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:16:16.920354 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7c23b6-72a4-477f-b585-19c386d88cc8 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:16:16.933756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1239/2453] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:16:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:16:17.102084 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0bc06-4bdd-47b7-896b-927ada3b6c65 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:16:17.111792 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0bc06-4bdd-47b7-896b-927ada3b6c65 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:16:17.116175 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde1aca-cd59-4126-8ad1-8adbc2d50fe9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:16:17.122653 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde1aca-cd59-4126-8ad1-8adbc2d50fe9 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:16:17.122653 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde1aca-cd59-4126-8ad1-8adbc2d50fe9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:16:17.122653 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afde1aca-cd59-4126-8ad1-8adbc2d50fe9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:16:17.122653 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-afde1aca-cd59-4126-8ad1-8adbc2d50fe9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:16:17.133469 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde1aca-cd59-4126-8ad1-8adbc2d50fe9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:16:17.134937 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afde1aca-cd59-4126-8ad1-8adbc2d50fe9 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:16:17.146138 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1215/2454] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:16:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:16:26.928764 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=26 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:16:27.141732 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=6 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:16:27.873978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aad451-4abb-4e30-b887-2f507157874b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:16:27.874107 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aad451-4abb-4e30-b887-2f507157874b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:16:27.874399 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aad451-4abb-4e30-b887-2f507157874b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:16:27.917354 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f7aad451-4abb-4e30-b887-2f507157874b None None] MFA Rules not processed for user `806f0c03435e4c5daeadaa4eb437d936`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:16:27.918701 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f7aad451-4abb-4e30-b887-2f507157874b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:16:27.978409 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1240/2455] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:16:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:16:27.990915 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde1aca-cd59-4126-8ad1-8adbc2d50fe9 service cinder] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:16:27.992444 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-afde1aca-cd59-4126-8ad1-8adbc2d50fe9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:16:28.006823 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afde1aca-cd59-4126-8ad1-8adbc2d50fe9 service cinder] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:16:28.012540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0abba-811d-417e-bfed-08550e4a2075 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:16:28.013076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0abba-811d-417e-bfed-08550e4a2075 service nova] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:16:28.013360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0abba-811d-417e-bfed-08550e4a2075 service nova] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:16:28.013637 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0abba-811d-417e-bfed-08550e4a2075 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:16:28.015449 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-51c0abba-811d-417e-bfed-08550e4a2075 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:16:28.026347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0abba-811d-417e-bfed-08550e4a2075 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:16:28.028022 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0abba-811d-417e-bfed-08550e4a2075 service nova] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:16:28.038299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1216/2456] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:16:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:16:29.052791 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aad451-4abb-4e30-b887-2f507157874b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:16:29.066984 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aad451-4abb-4e30-b887-2f507157874b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:16:29.073864 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024ad90-be80-42a6-a3e3-a7457f8ca35b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:16:29.074689 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024ad90-be80-42a6-a3e3-a7457f8ca35b service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:16:29.074977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024ad90-be80-42a6-a3e3-a7457f8ca35b service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:16:29.075259 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024ad90-be80-42a6-a3e3-a7457f8ca35b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:16:29.077146 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e024ad90-be80-42a6-a3e3-a7457f8ca35b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:16:29.159453 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024ad90-be80-42a6-a3e3-a7457f8ca35b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:16:29.161702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024ad90-be80-42a6-a3e3-a7457f8ca35b service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:16:29.181932 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1241/2457] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:16:29 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:16:38.033229 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=6 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:16:39.178354 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=75 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:16:45.546701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe9649-4e7a-47fd-9038-8a3b8e68b2ab None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:16:45.547237 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe9649-4e7a-47fd-9038-8a3b8e68b2ab None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:16:45.547237 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe9649-4e7a-47fd-9038-8a3b8e68b2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:16:45.597204 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-d0fe9649-4e7a-47fd-9038-8a3b8e68b2ab None None] MFA Rules not processed for user `ed8114b7a28741268fdff71c866dbc77`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:16:45.599044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-d0fe9649-4e7a-47fd-9038-8a3b8e68b2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:16:45.664805 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1217/2458] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 2 09:16:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:16:45.825641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024ad90-be80-42a6-a3e3-a7457f8ca35b service nova] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:16:45.827114 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e024ad90-be80-42a6-a3e3-a7457f8ca35b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:16:45.922946 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024ad90-be80-42a6-a3e3-a7457f8ca35b service nova] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:16:45.929502 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f5b8b-527b-404f-8a43-8bd05270f9d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff92ef3b9da74ccab3fbc43f852e3ad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:16:45.930294 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f5b8b-527b-404f-8a43-8bd05270f9d6 service swift] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:16:45.930708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f5b8b-527b-404f-8a43-8bd05270f9d6 service swift] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:16:45.931110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f5b8b-527b-404f-8a43-8bd05270f9d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:16:45.940646 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f5b8b-527b-404f-8a43-8bd05270f9d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:16:45.942711 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f5b8b-527b-404f-8a43-8bd05270f9d6 service swift] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:16:45.951737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1242/2459] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 2 09:16:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:16:55.661695 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=40 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:16:55.947974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=41 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:16:58.221604 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe9649-4e7a-47fd-9038-8a3b8e68b2ab None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:16:58.231967 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe9649-4e7a-47fd-9038-8a3b8e68b2ab None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:16:58.235596 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe224f11-9c95-4c36-a8c3-0664690a9e02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:16:58.236203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe224f11-9c95-4c36-a8c3-0664690a9e02 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:16:58.236485 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe224f11-9c95-4c36-a8c3-0664690a9e02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:16:58.236760 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe224f11-9c95-4c36-a8c3-0664690a9e02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:16:58.239287 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-fe224f11-9c95-4c36-a8c3-0664690a9e02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:16:58.249078 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe224f11-9c95-4c36-a8c3-0664690a9e02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:16:58.250545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe224f11-9c95-4c36-a8c3-0664690a9e02 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:16:58.259382 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1218/2460] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:16:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:17:03.843862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae043554-43ec-4bd4-a513-af2b269d52cb None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:17:03.843862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae043554-43ec-4bd4-a513-af2b269d52cb None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:17:03.843862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae043554-43ec-4bd4-a513-af2b269d52cb None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:17:03.845177 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1243/2461] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 2 09:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:17:03.852273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488316d-62ae-4e5c-ac57-5a04810a27e1 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:17:03.858281 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488316d-62ae-4e5c-ac57-5a04810a27e1 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:17:03.858281 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488316d-62ae-4e5c-ac57-5a04810a27e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:17:03.892811 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-5488316d-62ae-4e5c-ac57-5a04810a27e1 None None] MFA Rules not processed for user `3191d804e8dc4b1a85f490319833ec07`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:17:03.894659 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-5488316d-62ae-4e5c-ac57-5a04810a27e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:17:03.905733 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1219/2462] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 2 09:17:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:17:03.922558 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae043554-43ec-4bd4-a513-af2b269d52cb None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:17:03.924965 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ae043554-43ec-4bd4-a513-af2b269d52cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:17:03.985662 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae043554-43ec-4bd4-a513-af2b269d52cb None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:17:03.989963 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a83ad-b795-4c7c-a6d7-f030b4ab98d0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4abd1512ad224da896c6f2257f9602e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:17:03.990518 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a83ad-b795-4c7c-a6d7-f030b4ab98d0 service glance] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:17:03.990798 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a83ad-b795-4c7c-a6d7-f030b4ab98d0 service glance] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:17:03.991068 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a83ad-b795-4c7c-a6d7-f030b4ab98d0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:17:04.006256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a83ad-b795-4c7c-a6d7-f030b4ab98d0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:17:04.007700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a83ad-b795-4c7c-a6d7-f030b4ab98d0 service glance] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:17:04.019279 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1244/2463] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:17:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:17:04.239946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ee4a2-aef8-4638-a77f-cd327b656231 None None] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:17:04.240276 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ee4a2-aef8-4638-a77f-cd327b656231 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:17:04.240506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ee4a2-aef8-4638-a77f-cd327b656231 None None] PATH_INFO: `/` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:17:04.241874 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1220/2464] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 2 09:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:17:04.246584 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a5de1-44a6-411e-95d5-9bde00cc5f16 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:17:04.246874 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a5de1-44a6-411e-95d5-9bde00cc5f16 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:17:04.247453 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4a5de1-44a6-411e-95d5-9bde00cc5f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:17:04.285853 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-0b4a5de1-44a6-411e-95d5-9bde00cc5f16 None None] MFA Rules not processed for user `ddf6f4d84bb34d2f810a0f6cec36da4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:17:04.287110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0b4a5de1-44a6-411e-95d5-9bde00cc5f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:17:04.293940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1245/2465] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 2 09:17:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:17:04.306325 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ee4a2-aef8-4638-a77f-cd327b656231 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:17:04.308001 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-531ee4a2-aef8-4638-a77f-cd327b656231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:17:04.322377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ee4a2-aef8-4638-a77f-cd327b656231 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:17:04.326191 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a994d191-0b36-4845-a9aa-6644407864ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:17:04.327623 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994d191-0b36-4845-a9aa-6644407864ab service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:17:04.327623 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994d191-0b36-4845-a9aa-6644407864ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:17:04.327623 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a994d191-0b36-4845-a9aa-6644407864ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:17:04.334811 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a994d191-0b36-4845-a9aa-6644407864ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:17:04.336240 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a994d191-0b36-4845-a9aa-6644407864ab service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:17:04.344124 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1221/2466] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:17:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:17:04.431648 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe8963-de30-4eca-9707-876a212ce399 None None] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:17:04.431969 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe8963-de30-4eca-9707-876a212ce399 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:17:04.432251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe8963-de30-4eca-9707-876a212ce399 None None] PATH_INFO: `/` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:17:04.433633 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1246/2467] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 2 09:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 02 09:17:04.439535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3b8d2-cc7c-4a72-8767-b5550850c073 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:17:04.439870 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3b8d2-cc7c-4a72-8767-b5550850c073 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:17:04.440146 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3b8d2-cc7c-4a72-8767-b5550850c073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:17:04.459810 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-f7a3b8d2-cc7c-4a72-8767-b5550850c073 None None] MFA Rules not processed for user `3191d804e8dc4b1a85f490319833ec07`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:17:04.461022 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f7a3b8d2-cc7c-4a72-8767-b5550850c073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:17:04.470340 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1222/2468] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 2 09:17:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:17:04.481393 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe8963-de30-4eca-9707-876a212ce399 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:17:04.493210 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe8963-de30-4eca-9707-876a212ce399 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:17:04.496836 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080320c5-9012-4117-b622-0ac547963151 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:17:04.497448 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080320c5-9012-4117-b622-0ac547963151 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:17:04.497730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080320c5-9012-4117-b622-0ac547963151 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:17:04.498040 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080320c5-9012-4117-b622-0ac547963151 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:17:04.508434 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080320c5-9012-4117-b622-0ac547963151 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:17:04.509859 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080320c5-9012-4117-b622-0ac547963151 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:17:04.520619 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1247/2469] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:17:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:17:04.589664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3b8d2-cc7c-4a72-8767-b5550850c073 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:17:04.591072 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f7a3b8d2-cc7c-4a72-8767-b5550850c073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:17:04.659869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3b8d2-cc7c-4a72-8767-b5550850c073 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:17:04.665172 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655427a-d443-48b6-9a87-52b3441aef93 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad5cc9927374836b0cc74db85b686fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:17:04.665799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655427a-d443-48b6-9a87-52b3441aef93 service placement] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:17:04.666133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655427a-d443-48b6-9a87-52b3441aef93 service placement] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:17:04.666408 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655427a-d443-48b6-9a87-52b3441aef93 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:17:04.668254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f655427a-d443-48b6-9a87-52b3441aef93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:17:04.680345 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f655427a-d443-48b6-9a87-52b3441aef93 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:17:04.681928 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f655427a-d443-48b6-9a87-52b3441aef93 service placement] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:17:04.692519 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1223/2470] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:17:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:17:07.014184 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9233f-6099-49d4-95f5-21f2f8df4ad0 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:17:07.014184 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9233f-6099-49d4-95f5-21f2f8df4ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:17:07.014679 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e9233f-6099-49d4-95f5-21f2f8df4ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:17:07.036611 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-64e9233f-6099-49d4-95f5-21f2f8df4ad0 None None] MFA Rules not processed for user `ed8114b7a28741268fdff71c866dbc77`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:17:07.037813 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-64e9233f-6099-49d4-95f5-21f2f8df4ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:17:07.045006 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1248/2471] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 2 09:17:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:17:07.057306 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655427a-d443-48b6-9a87-52b3441aef93 service placement] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:17:07.068796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655427a-d443-48b6-9a87-52b3441aef93 service placement] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:17:07.073671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba8ade-4482-4505-8b63-2d6791297cd2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff92ef3b9da74ccab3fbc43f852e3ad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:17:07.074303 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba8ade-4482-4505-8b63-2d6791297cd2 service swift] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:17:07.074585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba8ade-4482-4505-8b63-2d6791297cd2 service swift] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:17:07.074859 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba8ade-4482-4505-8b63-2d6791297cd2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:17:07.083632 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba8ade-4482-4505-8b63-2d6791297cd2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:17:07.085200 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba8ade-4482-4505-8b63-2d6791297cd2 service swift] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:17:07.099475 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1224/2472] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 2 09:17:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:17:17.031073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=44 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:17:17.090053 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=62 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:17:28.525303 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf9629-fcaf-4499-842c-7ee552f15386 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:17:28.525798 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf9629-fcaf-4499-842c-7ee552f15386 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:17:28.525923 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecf9629-fcaf-4499-842c-7ee552f15386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:17:28.545020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-3ecf9629-fcaf-4499-842c-7ee552f15386 None None] MFA Rules not processed for user `ed8114b7a28741268fdff71c866dbc77`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:17:28.548307 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3ecf9629-fcaf-4499-842c-7ee552f15386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:17:28.553557 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1249/2473] 10.222.0.46 () {58 vars in 967 bytes} [Tue Aug 2 09:17:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:17:28.566659 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba8ade-4482-4505-8b63-2d6791297cd2 service swift] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:17:28.577121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba8ade-4482-4505-8b63-2d6791297cd2 service swift] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:17:28.582888 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f3c81-7015-4481-afb1-f9ca7c33ad24 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff92ef3b9da74ccab3fbc43f852e3ad0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:17:28.583496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f3c81-7015-4481-afb1-f9ca7c33ad24 service swift] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:17:28.587166 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f3c81-7015-4481-afb1-f9ca7c33ad24 service swift] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:17:28.587166 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f3c81-7015-4481-afb1-f9ca7c33ad24 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:17:28.592119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f3c81-7015-4481-afb1-f9ca7c33ad24 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:17:28.593697 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f3c81-7015-4481-afb1-f9ca7c33ad24 service swift] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:17:28.601821 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1225/2474] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 2 09:17:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 09:17:29.391412 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0547c8-6406-4887-8a0a-ae78d8b21937 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:17:29.391830 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0547c8-6406-4887-8a0a-ae78d8b21937 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:17:29.393276 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0547c8-6406-4887-8a0a-ae78d8b21937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:17:29.412617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-de0547c8-6406-4887-8a0a-ae78d8b21937 None None] MFA Rules not processed for user `02569c995a6d4b678933599f60ad3909`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:17:29.413822 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-de0547c8-6406-4887-8a0a-ae78d8b21937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:17:29.471915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1250/2475] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:17:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:17:38.598652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=6 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:17:39.464402 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=28 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:17:45.633449 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f3c81-7015-4481-afb1-f9ca7c33ad24 service swift] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:17:45.644652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f3c81-7015-4481-afb1-f9ca7c33ad24 service swift] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:17:45.650308 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c1c41-2506-4ba8-8d66-f308ea47bf98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:17:45.650862 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c1c41-2506-4ba8-8d66-f308ea47bf98 service nova] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:17:45.651585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c1c41-2506-4ba8-8d66-f308ea47bf98 service nova] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:17:45.651585 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c1c41-2506-4ba8-8d66-f308ea47bf98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:17:45.653251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-0a8c1c41-2506-4ba8-8d66-f308ea47bf98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:17:45.666463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c1c41-2506-4ba8-8d66-f308ea47bf98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:17:45.668072 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c1c41-2506-4ba8-8d66-f308ea47bf98 service nova] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:17:45.679439 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1226/2476] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:17:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:17:50.484988 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0547c8-6406-4887-8a0a-ae78d8b21937 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:17:50.499049 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0547c8-6406-4887-8a0a-ae78d8b21937 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:17:50.504820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968fd57-b1fc-4941-a9ba-2f45274c0499 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:17:50.505526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968fd57-b1fc-4941-a9ba-2f45274c0499 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:17:50.505934 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968fd57-b1fc-4941-a9ba-2f45274c0499 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:17:50.506283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968fd57-b1fc-4941-a9ba-2f45274c0499 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:17:50.508174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5968fd57-b1fc-4941-a9ba-2f45274c0499 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:17:50.528964 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968fd57-b1fc-4941-a9ba-2f45274c0499 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:17:50.530705 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5968fd57-b1fc-4941-a9ba-2f45274c0499 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:17:50.542893 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1251/2477] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:17:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:17:55.674652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=6 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:18:00.537807 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=6 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:18:36.204650 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c1c41-2506-4ba8-8d66-f308ea47bf98 service nova] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:36.214093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c1c41-2506-4ba8-8d66-f308ea47bf98 service nova] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:36.218574 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768b351-44f6-4c9b-9ce2-8399e6490ce4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:36.219165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768b351-44f6-4c9b-9ce2-8399e6490ce4 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:36.219445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768b351-44f6-4c9b-9ce2-8399e6490ce4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:36.219718 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f768b351-44f6-4c9b-9ce2-8399e6490ce4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:36.221427 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f768b351-44f6-4c9b-9ce2-8399e6490ce4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:36.233346 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768b351-44f6-4c9b-9ce2-8399e6490ce4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:36.234784 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f768b351-44f6-4c9b-9ce2-8399e6490ce4 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:36.244434 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1227/2478] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:18:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:18:40.805264 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968fd57-b1fc-4941-a9ba-2f45274c0499 service nova] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:40.819805 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5968fd57-b1fc-4941-a9ba-2f45274c0499 service nova] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:40.825402 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f246f8-f226-4ad6-ac7f-573c6c997e21 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad5cc9927374836b0cc74db85b686fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:40.826147 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f246f8-f226-4ad6-ac7f-573c6c997e21 service placement] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:40.826917 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f246f8-f226-4ad6-ac7f-573c6c997e21 service placement] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:40.827194 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f246f8-f226-4ad6-ac7f-573c6c997e21 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:40.829051 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-47f246f8-f226-4ad6-ac7f-573c6c997e21 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:40.841001 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f246f8-f226-4ad6-ac7f-573c6c997e21 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:40.843399 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f246f8-f226-4ad6-ac7f-573c6c997e21 service placement] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:40.853591 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1252/2479] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:18:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:18:40.961005 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768b351-44f6-4c9b-9ce2-8399e6490ce4 service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:40.962671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f768b351-44f6-4c9b-9ce2-8399e6490ce4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:40.979793 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f768b351-44f6-4c9b-9ce2-8399e6490ce4 service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:40.986738 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d076c2e-09bb-4910-bbd4-b645d5998674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:40.986943 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d076c2e-09bb-4910-bbd4-b645d5998674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:40.987224 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d076c2e-09bb-4910-bbd4-b645d5998674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:40.987500 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d076c2e-09bb-4910-bbd4-b645d5998674 admin admin] PATH_INFO: `/v3/users/806f0c03435e4c5daeadaa4eb437d936` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:40.988980 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d076c2e-09bb-4910-bbd4-b645d5998674 admin admin] RBAC: Authorizing `identity:delete_user(user_id=806f0c03435e4c5daeadaa4eb437d936)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:40.992675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d076c2e-09bb-4910-bbd4-b645d5998674 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:41.214620 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-7d076c2e-09bb-4910-bbd4-b645d5998674 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '806f0c03435e4c5daeadaa4eb437d936'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:41.222132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-7d076c2e-09bb-4910-bbd4-b645d5998674 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '806f0c03435e4c5daeadaa4eb437d936'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:41.222807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-7d076c2e-09bb-4910-bbd4-b645d5998674 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '806f0c03435e4c5daeadaa4eb437d936'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:41.308224 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-7d076c2e-09bb-4910-bbd4-b645d5998674 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '806f0c03435e4c5daeadaa4eb437d936'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:41.322864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-7d076c2e-09bb-4910-bbd4-b645d5998674 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '806f0c03435e4c5daeadaa4eb437d936'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:41.330023 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1228/2480] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:18:40 2022] DELETE /identity/v3/users/806f0c03435e4c5daeadaa4eb437d936 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:18:41.572149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f246f8-f226-4ad6-ac7f-573c6c997e21 service placement] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:41.587807 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f246f8-f226-4ad6-ac7f-573c6c997e21 service placement] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:41.647731 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d7ca68-9387-4bdd-ac5f-2863fccd1972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:41.648383 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7ca68-9387-4bdd-ac5f-2863fccd1972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:41.648667 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7ca68-9387-4bdd-ac5f-2863fccd1972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:41.648944 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d7ca68-9387-4bdd-ac5f-2863fccd1972 admin admin] PATH_INFO: `/v3/projects/b9df882a572a4bed8966779e50eb7e7c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:41.650278 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7ca68-9387-4bdd-ac5f-2863fccd1972 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9df882a572a4bed8966779e50eb7e7c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:41.652194 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d7ca68-9387-4bdd-ac5f-2863fccd1972 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:41.730934 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-29d7ca68-9387-4bdd-ac5f-2863fccd1972 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9df882a572a4bed8966779e50eb7e7c', 'request_id': 'req-29d7ca68-9387-4bdd-ac5f-2863fccd1972'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:41.737860 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-29d7ca68-9387-4bdd-ac5f-2863fccd1972 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9df882a572a4bed8966779e50eb7e7c', 'request_id': 'req-29d7ca68-9387-4bdd-ac5f-2863fccd1972'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:41.793507 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-29d7ca68-9387-4bdd-ac5f-2863fccd1972 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9df882a572a4bed8966779e50eb7e7c', 'request_id': 'req-29d7ca68-9387-4bdd-ac5f-2863fccd1972'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:41.805874 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-29d7ca68-9387-4bdd-ac5f-2863fccd1972 admin admin] The token cache is being invalidate because project b9df882a572a4bed8966779e50eb7e7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:18:41.806496 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-29d7ca68-9387-4bdd-ac5f-2863fccd1972 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:41.806832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-29d7ca68-9387-4bdd-ac5f-2863fccd1972 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:41.808623 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1253/2481] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:18:41 2022] DELETE /identity/v3/projects/b9df882a572a4bed8966779e50eb7e7c => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:18:41.813751 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c5144-0f16-44f8-aa7a-588945c236c7 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:41.814079 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c5144-0f16-44f8-aa7a-588945c236c7 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:41.814359 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c5144-0f16-44f8-aa7a-588945c236c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:41.834146 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-128c5144-0f16-44f8-aa7a-588945c236c7 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:18:41.835376 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-128c5144-0f16-44f8-aa7a-588945c236c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:41.895180 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1229/2482] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:18:41.903026 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaac0d-ec3a-4c23-9bd0-89e8e89d7c49 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:41.903314 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaac0d-ec3a-4c23-9bd0-89e8e89d7c49 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:41.903610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aaac0d-ec3a-4c23-9bd0-89e8e89d7c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:41.925690 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-79aaac0d-ec3a-4c23-9bd0-89e8e89d7c49 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:18:41.926929 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-79aaac0d-ec3a-4c23-9bd0-89e8e89d7c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:41.935925 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1254/2483] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:18:41.941548 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c5144-0f16-44f8-aa7a-588945c236c7 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:41.951787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c5144-0f16-44f8-aa7a-588945c236c7 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:41.956333 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892f8a5-09b8-4a61-840f-79291893c187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:41.956940 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892f8a5-09b8-4a61-840f-79291893c187 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:41.957224 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892f8a5-09b8-4a61-840f-79291893c187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:41.957498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892f8a5-09b8-4a61-840f-79291893c187 admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:41.958256 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3892f8a5-09b8-4a61-840f-79291893c187 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:18:41.958561 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3892f8a5-09b8-4a61-840f-79291893c187 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:41.960150 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3892f8a5-09b8-4a61-840f-79291893c187 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:41.972285 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1230/2484] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:18:41.976767 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaac0d-ec3a-4c23-9bd0-89e8e89d7c49 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:41.986693 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aaac0d-ec3a-4c23-9bd0-89e8e89d7c49 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:41.991213 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797c459-901d-454f-b512-9a893f8fe83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:41.991815 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797c459-901d-454f-b512-9a893f8fe83b admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:41.992090 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797c459-901d-454f-b512-9a893f8fe83b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:41.992357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797c459-901d-454f-b512-9a893f8fe83b admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:41.993039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797c459-901d-454f-b512-9a893f8fe83b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:41.994949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797c459-901d-454f-b512-9a893f8fe83b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:42.093150 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1255/2485] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:18:41 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:18:42.097362 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892f8a5-09b8-4a61-840f-79291893c187 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:42.107507 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892f8a5-09b8-4a61-840f-79291893c187 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:42.151082 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:18:42.164170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d1332-d561-4a51-92fd-7329c8dbaf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:42.164787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d1332-d561-4a51-92fd-7329c8dbaf22 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:42.165067 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d1332-d561-4a51-92fd-7329c8dbaf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:42.165341 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d1332-d561-4a51-92fd-7329c8dbaf22 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:42.166033 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52d1332-d561-4a51-92fd-7329c8dbaf22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:42.167922 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52d1332-d561-4a51-92fd-7329c8dbaf22 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:42.249070 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1231/2486] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:18:42 2022] POST /identity/v3/users => generated 287 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:18:42.253370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797c459-901d-454f-b512-9a893f8fe83b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:42.264531 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797c459-901d-454f-b512-9a893f8fe83b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:42.269355 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcefe36-bbaf-4e3c-aeba-84572c7bbcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:42.270014 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcefe36-bbaf-4e3c-aeba-84572c7bbcd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:42.270371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcefe36-bbaf-4e3c-aeba-84572c7bbcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:42.270709 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcefe36-bbaf-4e3c-aeba-84572c7bbcd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:42.271288 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcefe36-bbaf-4e3c-aeba-84572c7bbcd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:42.272900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcefe36-bbaf-4e3c-aeba-84572c7bbcd1 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:42.285059 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1256/2487] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:18:42.292495 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d1332-d561-4a51-92fd-7329c8dbaf22 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:42.302741 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d1332-d561-4a51-92fd-7329c8dbaf22 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:42.307675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ab94e-7951-41d7-a317-1b19f26ee2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:42.308286 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ab94e-7951-41d7-a317-1b19f26ee2f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:42.308565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ab94e-7951-41d7-a317-1b19f26ee2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:42.308837 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ab94e-7951-41d7-a317-1b19f26ee2f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:42.309523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ab94e-7951-41d7-a317-1b19f26ee2f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:42.311161 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ab94e-7951-41d7-a317-1b19f26ee2f4 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:42.322096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1232/2488] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:18:42.327163 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcefe36-bbaf-4e3c-aeba-84572c7bbcd1 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:42.337657 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcefe36-bbaf-4e3c-aeba-84572c7bbcd1 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:42.343421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6eb0b-00ec-4fd4-91da-00d74af5c1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:42.344023 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6eb0b-00ec-4fd4-91da-00d74af5c1cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:42.344423 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6eb0b-00ec-4fd4-91da-00d74af5c1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:42.344701 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab6eb0b-00ec-4fd4-91da-00d74af5c1cc admin admin] PATH_INFO: `/v3/projects/8b77eb0762b346d08d74fcffcb3b3550/users/6041d1b8a0434f599dc77d13ff862ddb/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:42.359874 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6eb0b-00ec-4fd4-91da-00d74af5c1cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b77eb0762b346d08d74fcffcb3b3550, user_id=6041d1b8a0434f599dc77d13ff862ddb, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:42.362983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab6eb0b-00ec-4fd4-91da-00d74af5c1cc admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:42.428065 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1257/2489] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:18:42 2022] PUT /identity/v3/projects/8b77eb0762b346d08d74fcffcb3b3550/users/6041d1b8a0434f599dc77d13ff862ddb/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:18:42.439768 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a18b1d-38aa-4caa-8255-afc9204d7e21 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:42.440055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a18b1d-38aa-4caa-8255-afc9204d7e21 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:42.440381 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a18b1d-38aa-4caa-8255-afc9204d7e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:42.469414 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-a2a18b1d-38aa-4caa-8255-afc9204d7e21 None None] MFA Rules not processed for user `6041d1b8a0434f599dc77d13ff862ddb`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:18:42.470718 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-a2a18b1d-38aa-4caa-8255-afc9204d7e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:42.540169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1233/2490] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:18:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:18:42.551569 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6eb0b-00ec-4fd4-91da-00d74af5c1cc admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:42.553284 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8ab6eb0b-00ec-4fd4-91da-00d74af5c1cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:42.568366 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:18:42.629053 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab6eb0b-00ec-4fd4-91da-00d74af5c1cc admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:42.634590 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6646e6-d361-42f8-9fa0-ae5dfefa8b8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:42.635216 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6646e6-d361-42f8-9fa0-ae5dfefa8b8c service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:42.635497 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6646e6-d361-42f8-9fa0-ae5dfefa8b8c service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:42.635769 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6646e6-d361-42f8-9fa0-ae5dfefa8b8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:42.644810 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6646e6-d361-42f8-9fa0-ae5dfefa8b8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:42.646630 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6646e6-d361-42f8-9fa0-ae5dfefa8b8c service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:42.656410 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1258/2491] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:18:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:18:42.863025 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a18b1d-38aa-4caa-8255-afc9204d7e21 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:42.864429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-a2a18b1d-38aa-4caa-8255-afc9204d7e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:42.921518 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a18b1d-38aa-4caa-8255-afc9204d7e21 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:42.925225 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41387ca3-6514-4b8e-8e1e-76a476d4832f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:42.925847 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41387ca3-6514-4b8e-8e1e-76a476d4832f service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:42.926159 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41387ca3-6514-4b8e-8e1e-76a476d4832f service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:42.926436 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41387ca3-6514-4b8e-8e1e-76a476d4832f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:42.928272 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-41387ca3-6514-4b8e-8e1e-76a476d4832f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:42.995604 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41387ca3-6514-4b8e-8e1e-76a476d4832f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:42.997052 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41387ca3-6514-4b8e-8e1e-76a476d4832f service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:43.007622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1234/2492] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:18:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:18:46.907856 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6646e6-d361-42f8-9fa0-ae5dfefa8b8c service nova] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:46.916532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6646e6-d361-42f8-9fa0-ae5dfefa8b8c service nova] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:46.927076 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900471d-598e-4cb7-a163-d5fb823909a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:46.927743 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900471d-598e-4cb7-a163-d5fb823909a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:46.928026 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900471d-598e-4cb7-a163-d5fb823909a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:46.928301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900471d-598e-4cb7-a163-d5fb823909a9 admin admin] PATH_INFO: `/v3/users/6041d1b8a0434f599dc77d13ff862ddb` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:46.929542 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a900471d-598e-4cb7-a163-d5fb823909a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6041d1b8a0434f599dc77d13ff862ddb)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:46.931604 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a900471d-598e-4cb7-a163-d5fb823909a9 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:46.959374 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41387ca3-6514-4b8e-8e1e-76a476d4832f service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:46.960915 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-41387ca3-6514-4b8e-8e1e-76a476d4832f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:46.970339 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:18:46.974233 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41387ca3-6514-4b8e-8e1e-76a476d4832f service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:46.978882 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e94fc-2ddc-47a5-a595-4cb8560f88ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:46.979513 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e94fc-2ddc-47a5-a595-4cb8560f88ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:46.979794 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e94fc-2ddc-47a5-a595-4cb8560f88ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:46.980236 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e94fc-2ddc-47a5-a595-4cb8560f88ef admin admin] PATH_INFO: `/v3/users/50918811a90a444eaf123b2aeb4a75b3` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:46.981290 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e94fc-2ddc-47a5-a595-4cb8560f88ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=50918811a90a444eaf123b2aeb4a75b3)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:46.984369 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e94fc-2ddc-47a5-a595-4cb8560f88ef admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:47.144270 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a900471d-598e-4cb7-a163-d5fb823909a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6041d1b8a0434f599dc77d13ff862ddb'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:47.151640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a900471d-598e-4cb7-a163-d5fb823909a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6041d1b8a0434f599dc77d13ff862ddb'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:47.152338 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a900471d-598e-4cb7-a163-d5fb823909a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6041d1b8a0434f599dc77d13ff862ddb'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:47.165305 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a900471d-598e-4cb7-a163-d5fb823909a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6041d1b8a0434f599dc77d13ff862ddb'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:47.168543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-a900471d-598e-4cb7-a163-d5fb823909a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6041d1b8a0434f599dc77d13ff862ddb'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:47.201135 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b25e94fc-2ddc-47a5-a595-4cb8560f88ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50918811a90a444eaf123b2aeb4a75b3'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:47.208248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b25e94fc-2ddc-47a5-a595-4cb8560f88ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50918811a90a444eaf123b2aeb4a75b3'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:47.208941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b25e94fc-2ddc-47a5-a595-4cb8560f88ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50918811a90a444eaf123b2aeb4a75b3'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:47.403950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1259/2493] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:18:46 2022] DELETE /identity/v3/users/6041d1b8a0434f599dc77d13ff862ddb => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:18:47.408606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f103b3-d7d8-433b-9803-ff095e49c7f3 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:47.408904 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f103b3-d7d8-433b-9803-ff095e49c7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:47.409191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f103b3-d7d8-433b-9803-ff095e49c7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:47.428054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-92f103b3-d7d8-433b-9803-ff095e49c7f3 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:18:47.429262 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-92f103b3-d7d8-433b-9803-ff095e49c7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:47.478638 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b25e94fc-2ddc-47a5-a595-4cb8560f88ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50918811a90a444eaf123b2aeb4a75b3'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:47.491106 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1260/2494] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:18:47.491694 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b25e94fc-2ddc-47a5-a595-4cb8560f88ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50918811a90a444eaf123b2aeb4a75b3'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:47.498417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1235/2495] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:18:46 2022] DELETE /identity/v3/users/50918811a90a444eaf123b2aeb4a75b3 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:18:47.501281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f103b3-d7d8-433b-9803-ff095e49c7f3 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:47.502760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-92f103b3-d7d8-433b-9803-ff095e49c7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:47.517594 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:18:47.562953 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f103b3-d7d8-433b-9803-ff095e49c7f3 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:47.614242 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb083e-75fc-4b35-a632-227ceb4c4af7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:47.614867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb083e-75fc-4b35-a632-227ceb4c4af7 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:47.615149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb083e-75fc-4b35-a632-227ceb4c4af7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:47.615424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb083e-75fc-4b35-a632-227ceb4c4af7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:47.617237 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f9fb083e-75fc-4b35-a632-227ceb4c4af7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:47.684465 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb083e-75fc-4b35-a632-227ceb4c4af7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:47.687586 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb083e-75fc-4b35-a632-227ceb4c4af7 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:47.695606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1261/2496] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:18:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:18:47.746605 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e94fc-2ddc-47a5-a595-4cb8560f88ef admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:47.748122 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-b25e94fc-2ddc-47a5-a595-4cb8560f88ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:47.762846 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e94fc-2ddc-47a5-a595-4cb8560f88ef admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:47.770604 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac24c9-2e71-4779-b200-87d3ccd74d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:47.775285 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac24c9-2e71-4779-b200-87d3ccd74d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:47.775285 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac24c9-2e71-4779-b200-87d3ccd74d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:47.775285 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac24c9-2e71-4779-b200-87d3ccd74d0e admin admin] PATH_INFO: `/v3/projects/7e47ffad695c4b049ca730c8cc43b9f9` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:47.775285 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ac24c9-2e71-4779-b200-87d3ccd74d0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e47ffad695c4b049ca730c8cc43b9f9)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:47.775285 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ac24c9-2e71-4779-b200-87d3ccd74d0e admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:47.905207 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b7ac24c9-2e71-4779-b200-87d3ccd74d0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e47ffad695c4b049ca730c8cc43b9f9', 'request_id': 'req-b7ac24c9-2e71-4779-b200-87d3ccd74d0e'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:47.911945 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b7ac24c9-2e71-4779-b200-87d3ccd74d0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e47ffad695c4b049ca730c8cc43b9f9', 'request_id': 'req-b7ac24c9-2e71-4779-b200-87d3ccd74d0e'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:47.918041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b7ac24c9-2e71-4779-b200-87d3ccd74d0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e47ffad695c4b049ca730c8cc43b9f9', 'request_id': 'req-b7ac24c9-2e71-4779-b200-87d3ccd74d0e'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:48.005872 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b7ac24c9-2e71-4779-b200-87d3ccd74d0e admin admin] The token cache is being invalidate because project 7e47ffad695c4b049ca730c8cc43b9f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:18:48.006580 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b7ac24c9-2e71-4779-b200-87d3ccd74d0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:48.007113 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b7ac24c9-2e71-4779-b200-87d3ccd74d0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:48.009121 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1236/2497] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:18:47 2022] DELETE /identity/v3/projects/7e47ffad695c4b049ca730c8cc43b9f9 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:18:48.014685 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd7cf7-79e0-4636-b3d4-e1437133f06b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:48.015298 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd7cf7-79e0-4636-b3d4-e1437133f06b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:48.015298 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd7cf7-79e0-4636-b3d4-e1437133f06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:48.035149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-eafd7cf7-79e0-4636-b3d4-e1437133f06b None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:18:48.036384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-eafd7cf7-79e0-4636-b3d4-e1437133f06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:48.098315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1262/2498] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:18:48.107659 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde51c2a-4778-41cb-9c08-3a7f97c22751 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:48.107975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde51c2a-4778-41cb-9c08-3a7f97c22751 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:48.108272 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde51c2a-4778-41cb-9c08-3a7f97c22751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:48.128367 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-dde51c2a-4778-41cb-9c08-3a7f97c22751 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:18:48.129561 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-dde51c2a-4778-41cb-9c08-3a7f97c22751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:48.137543 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1237/2499] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:18:48.143089 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd7cf7-79e0-4636-b3d4-e1437133f06b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:48.148566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde51c2a-4778-41cb-9c08-3a7f97c22751 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:48.150216 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-dde51c2a-4778-41cb-9c08-3a7f97c22751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:48.152929 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd7cf7-79e0-4636-b3d4-e1437133f06b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:48.160412 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746bcd9b-6a99-413c-ae71-724758def129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:48.160412 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bcd9b-6a99-413c-ae71-724758def129 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:48.160917 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bcd9b-6a99-413c-ae71-724758def129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:48.161201 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bcd9b-6a99-413c-ae71-724758def129 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:48.162316 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746bcd9b-6a99-413c-ae71-724758def129 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:18:48.162622 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746bcd9b-6a99-413c-ae71-724758def129 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:48.163550 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde51c2a-4778-41cb-9c08-3a7f97c22751 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:48.164243 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746bcd9b-6a99-413c-ae71-724758def129 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:48.168507 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7286526-fca8-4f59-93aa-c88730951f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:48.169220 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7286526-fca8-4f59-93aa-c88730951f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:48.169571 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7286526-fca8-4f59-93aa-c88730951f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:48.170552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7286526-fca8-4f59-93aa-c88730951f0e admin admin] PATH_INFO: `/v3/projects/8b77eb0762b346d08d74fcffcb3b3550` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:48.171851 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7286526-fca8-4f59-93aa-c88730951f0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b77eb0762b346d08d74fcffcb3b3550)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:48.173949 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7286526-fca8-4f59-93aa-c88730951f0e admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:48.183405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1263/2500] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:18:48.184183 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746bcd9b-6a99-413c-ae71-724758def129 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:48.191652 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:18:48.195434 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746bcd9b-6a99-413c-ae71-724758def129 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:48.200754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb71f62-c289-401f-aff6-107c8c551b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:48.206055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb71f62-c289-401f-aff6-107c8c551b32 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:48.206055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb71f62-c289-401f-aff6-107c8c551b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:48.206055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb71f62-c289-401f-aff6-107c8c551b32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:48.206055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb71f62-c289-401f-aff6-107c8c551b32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:48.206055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb71f62-c289-401f-aff6-107c8c551b32 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:48.329812 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=77,keystone:DELETE=21,keystone:INSERT=2 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:18:48.331204 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-d7286526-fca8-4f59-93aa-c88730951f0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b77eb0762b346d08d74fcffcb3b3550', 'request_id': 'req-d7286526-fca8-4f59-93aa-c88730951f0e'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:48.332528 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1264/2501] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:18:48 2022] POST /identity/v3/projects => generated 403 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:18:48.336713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb71f62-c289-401f-aff6-107c8c551b32 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:48.337720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-d7286526-fca8-4f59-93aa-c88730951f0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b77eb0762b346d08d74fcffcb3b3550', 'request_id': 'req-d7286526-fca8-4f59-93aa-c88730951f0e'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:48.344277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-d7286526-fca8-4f59-93aa-c88730951f0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b77eb0762b346d08d74fcffcb3b3550', 'request_id': 'req-d7286526-fca8-4f59-93aa-c88730951f0e'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:48.398750 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb71f62-c289-401f-aff6-107c8c551b32 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:48.403437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031a69c-8789-4b1c-9385-aa0d8a5e3c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:48.414091 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031a69c-8789-4b1c-9385-aa0d8a5e3c60 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:48.414091 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031a69c-8789-4b1c-9385-aa0d8a5e3c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:48.414091 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b031a69c-8789-4b1c-9385-aa0d8a5e3c60 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:48.414091 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b031a69c-8789-4b1c-9385-aa0d8a5e3c60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:48.414091 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b031a69c-8789-4b1c-9385-aa0d8a5e3c60 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:48.442706 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-d7286526-fca8-4f59-93aa-c88730951f0e admin admin] The token cache is being invalidate because project 8b77eb0762b346d08d74fcffcb3b3550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:18:48.442706 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-d7286526-fca8-4f59-93aa-c88730951f0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:48.442706 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-d7286526-fca8-4f59-93aa-c88730951f0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:48.442706 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1238/2502] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:18:48 2022] DELETE /identity/v3/projects/8b77eb0762b346d08d74fcffcb3b3550 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:18:48.445960 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2bde6-f3bf-4775-a51e-f56d27a3ce34 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:48.446247 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2bde6-f3bf-4775-a51e-f56d27a3ce34 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:48.446523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2bde6-f3bf-4775-a51e-f56d27a3ce34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:48.466561 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-01d2bde6-f3bf-4775-a51e-f56d27a3ce34 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:18:48.469404 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-01d2bde6-f3bf-4775-a51e-f56d27a3ce34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:48.477562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1239/2503] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:18:48.484972 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7629a11-433e-43f2-9d9c-78661040752e None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:48.485261 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7629a11-433e-43f2-9d9c-78661040752e None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:48.485558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7629a11-433e-43f2-9d9c-78661040752e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:48.505708 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-e7629a11-433e-43f2-9d9c-78661040752e None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:18:48.507077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e7629a11-433e-43f2-9d9c-78661040752e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:48.515794 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1240/2504] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:18:48.520166 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7629a11-433e-43f2-9d9c-78661040752e None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:48.529958 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7629a11-433e-43f2-9d9c-78661040752e None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:48.534847 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ae0e4-1310-4db4-9351-13f9ac02982e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:48.535407 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ae0e4-1310-4db4-9351-13f9ac02982e admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:48.535691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ae0e4-1310-4db4-9351-13f9ac02982e admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:48.535970 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ae0e4-1310-4db4-9351-13f9ac02982e admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:48.536670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ae0e4-1310-4db4-9351-13f9ac02982e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:18:48.536975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ae0e4-1310-4db4-9351-13f9ac02982e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:48.537638 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1265/2505] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:18:48 2022] POST /identity/v3/users => generated 302 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:18:48.538636 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ae0e4-1310-4db4-9351-13f9ac02982e admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:48.541386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031a69c-8789-4b1c-9385-aa0d8a5e3c60 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:48.543086 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b031a69c-8789-4b1c-9385-aa0d8a5e3c60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:48.554098 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1241/2506] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:18:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:18:48.555106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ae0e4-1310-4db4-9351-13f9ac02982e admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:48.555263 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b031a69c-8789-4b1c-9385-aa0d8a5e3c60 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:48.560122 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525bcdd-af55-4f4d-a338-28e57fb04d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:48.561231 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525bcdd-af55-4f4d-a338-28e57fb04d0a admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:48.567176 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525bcdd-af55-4f4d-a338-28e57fb04d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:48.567176 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525bcdd-af55-4f4d-a338-28e57fb04d0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:48.567176 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3525bcdd-af55-4f4d-a338-28e57fb04d0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:48.567176 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3525bcdd-af55-4f4d-a338-28e57fb04d0a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:48.567581 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ae0e4-1310-4db4-9351-13f9ac02982e admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:48.571838 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd4716-4f48-4b8e-b8d8-5a7c5b34ca28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:48.572445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd4716-4f48-4b8e-b8d8-5a7c5b34ca28 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:48.572729 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd4716-4f48-4b8e-b8d8-5a7c5b34ca28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:48.573026 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd4716-4f48-4b8e-b8d8-5a7c5b34ca28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:48.573695 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd4716-4f48-4b8e-b8d8-5a7c5b34ca28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:48.574788 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1266/2507] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:18:48.575630 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafd4716-4f48-4b8e-b8d8-5a7c5b34ca28 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:48.579369 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525bcdd-af55-4f4d-a338-28e57fb04d0a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:48.589356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525bcdd-af55-4f4d-a338-28e57fb04d0a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:48.594672 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfaf3c-134d-4dcd-a629-a4dd456f588c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:48.595381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfaf3c-134d-4dcd-a629-a4dd456f588c admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:48.595668 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfaf3c-134d-4dcd-a629-a4dd456f588c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:48.595990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfaf3c-134d-4dcd-a629-a4dd456f588c admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:48.596665 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bfaf3c-134d-4dcd-a629-a4dd456f588c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:48.598331 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bfaf3c-134d-4dcd-a629-a4dd456f588c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:48.609372 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1267/2508] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:18:48.613508 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfaf3c-134d-4dcd-a629-a4dd456f588c admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:48.624979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfaf3c-134d-4dcd-a629-a4dd456f588c admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:48.630230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1644a974-e62e-448c-9fbd-5de18336b7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:48.630398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644a974-e62e-448c-9fbd-5de18336b7cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:48.630618 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644a974-e62e-448c-9fbd-5de18336b7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:48.630894 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1644a974-e62e-448c-9fbd-5de18336b7cd admin admin] PATH_INFO: `/v3/projects/1fd59d5bffee4bfb86891ad8e9889b62/users/62d199d7aa21452ea04b70d38b3011ae/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:48.646364 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1644a974-e62e-448c-9fbd-5de18336b7cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fd59d5bffee4bfb86891ad8e9889b62, user_id=62d199d7aa21452ea04b70d38b3011ae, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:48.649029 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1644a974-e62e-448c-9fbd-5de18336b7cd admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:48.656501 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1242/2509] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:18:48 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:18:48.660832 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd4716-4f48-4b8e-b8d8-5a7c5b34ca28 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:48.723900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd4716-4f48-4b8e-b8d8-5a7c5b34ca28 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:48.728622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0592c-9cc3-4413-a4e4-e07cb28750db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:48.733500 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0592c-9cc3-4413-a4e4-e07cb28750db admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:48.733500 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0592c-9cc3-4413-a4e4-e07cb28750db admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:48.733500 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe0592c-9cc3-4413-a4e4-e07cb28750db admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:48.733500 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0592c-9cc3-4413-a4e4-e07cb28750db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:48.733500 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe0592c-9cc3-4413-a4e4-e07cb28750db admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:48.736346 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1268/2510] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:18:48 2022] PUT /identity/v3/projects/1fd59d5bffee4bfb86891ad8e9889b62/users/62d199d7aa21452ea04b70d38b3011ae/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:18:48.741409 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e8d22-ee49-4e98-ad6d-5dde7b7c14da None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:48.741695 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e8d22-ee49-4e98-ad6d-5dde7b7c14da None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:48.742007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e8d22-ee49-4e98-ad6d-5dde7b7c14da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:48.761140 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-333e8d22-ee49-4e98-ad6d-5dde7b7c14da None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:18:48.762380 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-333e8d22-ee49-4e98-ad6d-5dde7b7c14da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:48.816374 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1243/2511] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:18:48 2022] POST /identity/v3/users => generated 283 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:18:48.820602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0592c-9cc3-4413-a4e4-e07cb28750db admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:48.824465 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1269/2512] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:18:48.834089 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe0592c-9cc3-4413-a4e4-e07cb28750db admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:48.834588 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e8d22-ee49-4e98-ad6d-5dde7b7c14da None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:48.835989 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-333e8d22-ee49-4e98-ad6d-5dde7b7c14da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:48.839119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0df98-5918-4781-9ccd-e4bf1e61d43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:48.839776 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0df98-5918-4781-9ccd-e4bf1e61d43e admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:48.840061 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0df98-5918-4781-9ccd-e4bf1e61d43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:48.840340 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0df98-5918-4781-9ccd-e4bf1e61d43e admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:48.841293 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b0df98-5918-4781-9ccd-e4bf1e61d43e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:48.844128 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b0df98-5918-4781-9ccd-e4bf1e61d43e admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:48.854532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1244/2513] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:18:48.859358 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0df98-5918-4781-9ccd-e4bf1e61d43e admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:48.868887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:18:48.869125 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0df98-5918-4781-9ccd-e4bf1e61d43e admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:48.876394 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c7f55-a7e4-4de4-a524-270a97228b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:48.877307 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c7f55-a7e4-4de4-a524-270a97228b0b admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:48.877664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c7f55-a7e4-4de4-a524-270a97228b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:48.877966 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41c7f55-a7e4-4de4-a524-270a97228b0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:48.878722 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c7f55-a7e4-4de4-a524-270a97228b0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:48.883279 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41c7f55-a7e4-4de4-a524-270a97228b0b admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:48.892650 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1245/2514] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:18:48.896697 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c7f55-a7e4-4de4-a524-270a97228b0b admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:48.896855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e8d22-ee49-4e98-ad6d-5dde7b7c14da None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:48.900629 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a254df-3536-4281-9478-f23a1575a1c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:48.901513 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a254df-3536-4281-9478-f23a1575a1c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:48.901975 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a254df-3536-4281-9478-f23a1575a1c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:48.903186 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a254df-3536-4281-9478-f23a1575a1c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:48.907345 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41c7f55-a7e4-4de4-a524-270a97228b0b admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:48.915671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7149c5-da2b-40cf-ad26-fcc1715661c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:48.915671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7149c5-da2b-40cf-ad26-fcc1715661c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:48.915671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7149c5-da2b-40cf-ad26-fcc1715661c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:48.915671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7149c5-da2b-40cf-ad26-fcc1715661c8 admin admin] PATH_INFO: `/v3/projects/2a776c2f8e6b46578d1d9908947d2f66/users/d74394a975214a328cbd054465ec45aa/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:48.916307 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a254df-3536-4281-9478-f23a1575a1c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:48.916307 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a254df-3536-4281-9478-f23a1575a1c1 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:48.925195 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1270/2515] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:18:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:18:48.934527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7149c5-da2b-40cf-ad26-fcc1715661c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a776c2f8e6b46578d1d9908947d2f66, user_id=d74394a975214a328cbd054465ec45aa, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:48.937253 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7149c5-da2b-40cf-ad26-fcc1715661c8 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:49.045112 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1246/2516] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:18:48 2022] PUT /identity/v3/projects/2a776c2f8e6b46578d1d9908947d2f66/users/d74394a975214a328cbd054465ec45aa/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:18:49.055592 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25810b8a-cd84-4c5b-9be6-f4d67c9a9b82 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:49.056111 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25810b8a-cd84-4c5b-9be6-f4d67c9a9b82 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:49.056258 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25810b8a-cd84-4c5b-9be6-f4d67c9a9b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:49.085831 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-25810b8a-cd84-4c5b-9be6-f4d67c9a9b82 None None] MFA Rules not processed for user `d74394a975214a328cbd054465ec45aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:18:49.089054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-25810b8a-cd84-4c5b-9be6-f4d67c9a9b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:49.155228 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1271/2517] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:18:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:18:49.166251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7149c5-da2b-40cf-ad26-fcc1715661c8 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:49.167940 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-7c7149c5-da2b-40cf-ad26-fcc1715661c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:49.202448 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=91,keystone:DELETE=3,keystone:INSERT=6 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:18:49.226068 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7149c5-da2b-40cf-ad26-fcc1715661c8 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:49.229823 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158187c-20c8-4924-8bd1-e805560ac3c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:49.230518 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158187c-20c8-4924-8bd1-e805560ac3c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:49.230802 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158187c-20c8-4924-8bd1-e805560ac3c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:49.231079 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158187c-20c8-4924-8bd1-e805560ac3c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:49.240096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e158187c-20c8-4924-8bd1-e805560ac3c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:49.241539 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e158187c-20c8-4924-8bd1-e805560ac3c0 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:49.251398 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1247/2518] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:18:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:18:50.268872 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73198455-3d4e-4a59-8f33-ae93cae12458 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:50.269431 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73198455-3d4e-4a59-8f33-ae93cae12458 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:50.269431 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73198455-3d4e-4a59-8f33-ae93cae12458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:50.301273 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-73198455-3d4e-4a59-8f33-ae93cae12458 None None] MFA Rules not processed for user `62d199d7aa21452ea04b70d38b3011ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:18:50.306817 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-73198455-3d4e-4a59-8f33-ae93cae12458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:50.371849 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1272/2519] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:18:50 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:18:50.376378 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158187c-20c8-4924-8bd1-e805560ac3c0 service cinder] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:50.439356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158187c-20c8-4924-8bd1-e805560ac3c0 service cinder] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:50.443850 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bba5dc-64cf-495b-a432-36528d9787c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:50.444465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bba5dc-64cf-495b-a432-36528d9787c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:50.444749 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bba5dc-64cf-495b-a432-36528d9787c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:50.445022 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bba5dc-64cf-495b-a432-36528d9787c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:50.445692 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bba5dc-64cf-495b-a432-36528d9787c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:50.447626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bba5dc-64cf-495b-a432-36528d9787c1 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:50.559128 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1248/2520] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:18:50 2022] POST /identity/v3/projects => generated 405 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:18:50.563412 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73198455-3d4e-4a59-8f33-ae93cae12458 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:50.574522 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73198455-3d4e-4a59-8f33-ae93cae12458 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:50.620495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:18:50.632991 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e3ba0-92d9-4543-b8f7-5d0c5364b98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:50.633623 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e3ba0-92d9-4543-b8f7-5d0c5364b98f admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:50.633933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e3ba0-92d9-4543-b8f7-5d0c5364b98f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:50.634221 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e3ba0-92d9-4543-b8f7-5d0c5364b98f admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:50.634936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e3ba0-92d9-4543-b8f7-5d0c5364b98f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:50.636899 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e3ba0-92d9-4543-b8f7-5d0c5364b98f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:50.713364 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1273/2521] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:18:50 2022] POST /identity/v3/users => generated 309 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:18:50.718188 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bba5dc-64cf-495b-a432-36528d9787c1 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:50.728966 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bba5dc-64cf-495b-a432-36528d9787c1 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:50.733838 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d3df7-bccb-4d28-8e58-f3c2e0498d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:50.734474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d3df7-bccb-4d28-8e58-f3c2e0498d7f admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:50.734754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d3df7-bccb-4d28-8e58-f3c2e0498d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:50.735029 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5d3df7-bccb-4d28-8e58-f3c2e0498d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:50.735712 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d3df7-bccb-4d28-8e58-f3c2e0498d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:50.737292 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5d3df7-bccb-4d28-8e58-f3c2e0498d7f admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:50.750105 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1249/2522] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:18:50.754526 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e3ba0-92d9-4543-b8f7-5d0c5364b98f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:50.765209 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e3ba0-92d9-4543-b8f7-5d0c5364b98f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:50.769718 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e41c9-72db-45c8-bf72-f758f73186f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:50.770406 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e41c9-72db-45c8-bf72-f758f73186f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:50.770684 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e41c9-72db-45c8-bf72-f758f73186f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:50.770954 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e41c9-72db-45c8-bf72-f758f73186f8 admin admin] PATH_INFO: `/v3/projects/805f4f91a6c949f097beaade5ad587ce/users/4f7f2efa9122495cae904fdfef5a44dc/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:50.792448 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e41c9-72db-45c8-bf72-f758f73186f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=805f4f91a6c949f097beaade5ad587ce, user_id=4f7f2efa9122495cae904fdfef5a44dc, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:50.795393 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e41c9-72db-45c8-bf72-f758f73186f8 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:50.897269 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1274/2523] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:18:50 2022] PUT /identity/v3/projects/805f4f91a6c949f097beaade5ad587ce/users/4f7f2efa9122495cae904fdfef5a44dc/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:18:50.901552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d3df7-bccb-4d28-8e58-f3c2e0498d7f admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:50.912215 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5d3df7-bccb-4d28-8e58-f3c2e0498d7f admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:50.974675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3261638-c5f7-4977-834f-060289c609c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:50.975313 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3261638-c5f7-4977-834f-060289c609c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:50.975603 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3261638-c5f7-4977-834f-060289c609c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:50.975885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3261638-c5f7-4977-834f-060289c609c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:50.976587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3261638-c5f7-4977-834f-060289c609c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:50.978213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3261638-c5f7-4977-834f-060289c609c1 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:50.989461 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1250/2524] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:18:50.993520 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e41c9-72db-45c8-bf72-f758f73186f8 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:51.004650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e41c9-72db-45c8-bf72-f758f73186f8 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:51.009546 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cffde1-7547-4ae7-b816-bc441b458585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:51.015140 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cffde1-7547-4ae7-b816-bc441b458585 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:51.015140 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cffde1-7547-4ae7-b816-bc441b458585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:51.015140 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cffde1-7547-4ae7-b816-bc441b458585 admin admin] PATH_INFO: `/v3/projects/805f4f91a6c949f097beaade5ad587ce/users/4f7f2efa9122495cae904fdfef5a44dc/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:51.015140 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cffde1-7547-4ae7-b816-bc441b458585 admin admin] RBAC: Authorizing `identity:create_grant(project_id=805f4f91a6c949f097beaade5ad587ce, user_id=4f7f2efa9122495cae904fdfef5a44dc, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:51.016928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cffde1-7547-4ae7-b816-bc441b458585 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:51.027406 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1275/2525] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:18:50 2022] PUT /identity/v3/projects/805f4f91a6c949f097beaade5ad587ce/users/4f7f2efa9122495cae904fdfef5a44dc/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:18:52.277583 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38021b81-f1dc-4a0d-b9dd-941b7b6763e7 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:52.278391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38021b81-f1dc-4a0d-b9dd-941b7b6763e7 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:52.278391 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38021b81-f1dc-4a0d-b9dd-941b7b6763e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:52.312408 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-38021b81-f1dc-4a0d-b9dd-941b7b6763e7 None None] MFA Rules not processed for user `4f7f2efa9122495cae904fdfef5a44dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:18:52.313942 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-38021b81-f1dc-4a0d-b9dd-941b7b6763e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:52.348946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:18:52.400130 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1251/2526] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:18:52 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:18:53.730943 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cffde1-7547-4ae7-b816-bc441b458585 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:53.801127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cffde1-7547-4ae7-b816-bc441b458585 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:53.806135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fad8f5-1707-412d-83ea-0bf8259bab88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:53.806723 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fad8f5-1707-412d-83ea-0bf8259bab88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:53.807364 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fad8f5-1707-412d-83ea-0bf8259bab88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:53.807635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fad8f5-1707-412d-83ea-0bf8259bab88 admin admin] PATH_INFO: `/v3/users/62d199d7aa21452ea04b70d38b3011ae` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:53.808810 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fad8f5-1707-412d-83ea-0bf8259bab88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62d199d7aa21452ea04b70d38b3011ae)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:53.811315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fad8f5-1707-412d-83ea-0bf8259bab88 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:54.024794 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-93fad8f5-1707-412d-83ea-0bf8259bab88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62d199d7aa21452ea04b70d38b3011ae'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:54.032026 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-93fad8f5-1707-412d-83ea-0bf8259bab88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62d199d7aa21452ea04b70d38b3011ae'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:54.032886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-93fad8f5-1707-412d-83ea-0bf8259bab88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62d199d7aa21452ea04b70d38b3011ae'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:54.046054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-93fad8f5-1707-412d-83ea-0bf8259bab88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62d199d7aa21452ea04b70d38b3011ae'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:54.046737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-93fad8f5-1707-412d-83ea-0bf8259bab88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62d199d7aa21452ea04b70d38b3011ae'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:54.117578 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1276/2527] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:18:53 2022] DELETE /identity/v3/users/62d199d7aa21452ea04b70d38b3011ae => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:18:54.364895 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38021b81-f1dc-4a0d-b9dd-941b7b6763e7 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:54.376499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38021b81-f1dc-4a0d-b9dd-941b7b6763e7 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:54.433217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da98b04-936d-4562-b99b-67bf05c0c84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:54.433869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da98b04-936d-4562-b99b-67bf05c0c84f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:54.434352 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da98b04-936d-4562-b99b-67bf05c0c84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:54.434554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da98b04-936d-4562-b99b-67bf05c0c84f admin admin] PATH_INFO: `/v3/projects/1fd59d5bffee4bfb86891ad8e9889b62` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:54.435807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da98b04-936d-4562-b99b-67bf05c0c84f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fd59d5bffee4bfb86891ad8e9889b62)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:54.438426 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da98b04-936d-4562-b99b-67bf05c0c84f admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:54.564544 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9da98b04-936d-4562-b99b-67bf05c0c84f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fd59d5bffee4bfb86891ad8e9889b62', 'request_id': 'req-9da98b04-936d-4562-b99b-67bf05c0c84f'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:54.570824 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9da98b04-936d-4562-b99b-67bf05c0c84f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fd59d5bffee4bfb86891ad8e9889b62', 'request_id': 'req-9da98b04-936d-4562-b99b-67bf05c0c84f'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:54.576623 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9da98b04-936d-4562-b99b-67bf05c0c84f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fd59d5bffee4bfb86891ad8e9889b62', 'request_id': 'req-9da98b04-936d-4562-b99b-67bf05c0c84f'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:54.655864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9da98b04-936d-4562-b99b-67bf05c0c84f admin admin] The token cache is being invalidate because project 1fd59d5bffee4bfb86891ad8e9889b62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:18:54.656105 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9da98b04-936d-4562-b99b-67bf05c0c84f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:54.656708 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9da98b04-936d-4562-b99b-67bf05c0c84f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:54.658253 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1252/2528] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:18:54 2022] DELETE /identity/v3/projects/1fd59d5bffee4bfb86891ad8e9889b62 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:18:54.662341 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fad8f5-1707-412d-83ea-0bf8259bab88 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:54.663795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-93fad8f5-1707-412d-83ea-0bf8259bab88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:54.677574 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fad8f5-1707-412d-83ea-0bf8259bab88 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:54.694379 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:18:54.738974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7a0c44-f9c4-40df-9c9f-5b24d2151684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:54.739759 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a0c44-f9c4-40df-9c9f-5b24d2151684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:54.740068 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a0c44-f9c4-40df-9c9f-5b24d2151684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:54.740355 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a0c44-f9c4-40df-9c9f-5b24d2151684 admin admin] PATH_INFO: `/v3/users/4f7f2efa9122495cae904fdfef5a44dc` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:54.741617 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7a0c44-f9c4-40df-9c9f-5b24d2151684 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f7f2efa9122495cae904fdfef5a44dc)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:54.743581 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7a0c44-f9c4-40df-9c9f-5b24d2151684 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:54.897715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-3c7a0c44-f9c4-40df-9c9f-5b24d2151684 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f7f2efa9122495cae904fdfef5a44dc'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:54.902183 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-3c7a0c44-f9c4-40df-9c9f-5b24d2151684 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f7f2efa9122495cae904fdfef5a44dc'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:54.903204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-3c7a0c44-f9c4-40df-9c9f-5b24d2151684 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f7f2efa9122495cae904fdfef5a44dc'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:54.917332 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-3c7a0c44-f9c4-40df-9c9f-5b24d2151684 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f7f2efa9122495cae904fdfef5a44dc'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:54.921051 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-3c7a0c44-f9c4-40df-9c9f-5b24d2151684 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f7f2efa9122495cae904fdfef5a44dc'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:55.020286 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1277/2529] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:18:54 2022] DELETE /identity/v3/users/4f7f2efa9122495cae904fdfef5a44dc => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:18:55.287892 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da98b04-936d-4562-b99b-67bf05c0c84f admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:55.289417 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-9da98b04-936d-4562-b99b-67bf05c0c84f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:55.302688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da98b04-936d-4562-b99b-67bf05c0c84f admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:55.360756 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebbbba-4643-4153-b6cc-d6f11f83c21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:55.361393 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebbbba-4643-4153-b6cc-d6f11f83c21c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:55.363235 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebbbba-4643-4153-b6cc-d6f11f83c21c admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:55.363235 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebbbba-4643-4153-b6cc-d6f11f83c21c admin admin] PATH_INFO: `/v3/projects/805f4f91a6c949f097beaade5ad587ce` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:55.363235 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cebbbba-4643-4153-b6cc-d6f11f83c21c admin admin] RBAC: Authorizing `identity:delete_project(project_id=805f4f91a6c949f097beaade5ad587ce)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:55.364998 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cebbbba-4643-4153-b6cc-d6f11f83c21c admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:55.366486 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:18:55.505045 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-5cebbbba-4643-4153-b6cc-d6f11f83c21c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '805f4f91a6c949f097beaade5ad587ce', 'request_id': 'req-5cebbbba-4643-4153-b6cc-d6f11f83c21c'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:55.511104 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-5cebbbba-4643-4153-b6cc-d6f11f83c21c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '805f4f91a6c949f097beaade5ad587ce', 'request_id': 'req-5cebbbba-4643-4153-b6cc-d6f11f83c21c'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:55.517162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-5cebbbba-4643-4153-b6cc-d6f11f83c21c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '805f4f91a6c949f097beaade5ad587ce', 'request_id': 'req-5cebbbba-4643-4153-b6cc-d6f11f83c21c'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:55.603496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-5cebbbba-4643-4153-b6cc-d6f11f83c21c admin admin] The token cache is being invalidate because project 805f4f91a6c949f097beaade5ad587ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:18:55.603887 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-5cebbbba-4643-4153-b6cc-d6f11f83c21c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:55.604604 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-5cebbbba-4643-4153-b6cc-d6f11f83c21c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:18:55.607424 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1253/2530] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:18:55 2022] DELETE /identity/v3/projects/805f4f91a6c949f097beaade5ad587ce => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:18:55.611480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a11e03-6d42-499c-a418-e50e7093bd32 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:55.611775 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a11e03-6d42-499c-a418-e50e7093bd32 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:55.612060 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a11e03-6d42-499c-a418-e50e7093bd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:55.631972 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-88a11e03-6d42-499c-a418-e50e7093bd32 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:18:55.633178 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-88a11e03-6d42-499c-a418-e50e7093bd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:55.694007 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1278/2531] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:18:55.703597 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258eac3-bfc0-47f2-9f10-ad2f48d0f044 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:55.703885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258eac3-bfc0-47f2-9f10-ad2f48d0f044 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:55.704165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258eac3-bfc0-47f2-9f10-ad2f48d0f044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:55.724063 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-8258eac3-bfc0-47f2-9f10-ad2f48d0f044 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:18:55.725304 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-8258eac3-bfc0-47f2-9f10-ad2f48d0f044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:55.733784 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1254/2532] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:18:55.738205 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a11e03-6d42-499c-a418-e50e7093bd32 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:55.748504 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a11e03-6d42-499c-a418-e50e7093bd32 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:55.753270 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f784314-7156-401d-b9a4-d5643412348d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:55.753983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f784314-7156-401d-b9a4-d5643412348d admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:55.754264 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f784314-7156-401d-b9a4-d5643412348d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:55.754539 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f784314-7156-401d-b9a4-d5643412348d admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:55.755287 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f784314-7156-401d-b9a4-d5643412348d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:18:55.755540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f784314-7156-401d-b9a4-d5643412348d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:55.757143 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f784314-7156-401d-b9a4-d5643412348d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:55.768933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1279/2533] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:18:55.775850 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258eac3-bfc0-47f2-9f10-ad2f48d0f044 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:55.792694 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258eac3-bfc0-47f2-9f10-ad2f48d0f044 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:55.798149 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ac358-0a68-40ee-ad2b-870b1f7a2389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:55.801360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ac358-0a68-40ee-ad2b-870b1f7a2389 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:55.801360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ac358-0a68-40ee-ad2b-870b1f7a2389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:55.801360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ac358-0a68-40ee-ad2b-870b1f7a2389 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:55.801360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ac358-0a68-40ee-ad2b-870b1f7a2389 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:55.801905 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ac358-0a68-40ee-ad2b-870b1f7a2389 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:55.944228 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1255/2534] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:18:55 2022] POST /identity/v3/projects => generated 381 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:18:55.948450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f784314-7156-401d-b9a4-d5643412348d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:55.958781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f784314-7156-401d-b9a4-d5643412348d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:55.983839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:18:56.015426 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce522044-9840-4e39-8a03-5ca21cb2af3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:56.016005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce522044-9840-4e39-8a03-5ca21cb2af3b admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:56.016288 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce522044-9840-4e39-8a03-5ca21cb2af3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:56.016564 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce522044-9840-4e39-8a03-5ca21cb2af3b admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:56.017239 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce522044-9840-4e39-8a03-5ca21cb2af3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:56.019336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce522044-9840-4e39-8a03-5ca21cb2af3b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:56.124551 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1280/2535] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:18:55 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:18:56.128807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ac358-0a68-40ee-ad2b-870b1f7a2389 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:56.141129 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ac358-0a68-40ee-ad2b-870b1f7a2389 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:56.146507 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2a120f-9327-4f02-9777-ffd3234c516c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:56.147129 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a120f-9327-4f02-9777-ffd3234c516c admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:56.147411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a120f-9327-4f02-9777-ffd3234c516c admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:56.147688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2a120f-9327-4f02-9777-ffd3234c516c admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:56.148385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2a120f-9327-4f02-9777-ffd3234c516c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:56.150021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2a120f-9327-4f02-9777-ffd3234c516c admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:56.161382 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1256/2536] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:18:56.165465 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce522044-9840-4e39-8a03-5ca21cb2af3b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:56.176836 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce522044-9840-4e39-8a03-5ca21cb2af3b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:56.182841 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d374e-de9b-425c-b33e-072d425ae700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:56.184300 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d374e-de9b-425c-b33e-072d425ae700 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:56.184580 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d374e-de9b-425c-b33e-072d425ae700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:56.184877 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d374e-de9b-425c-b33e-072d425ae700 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:56.185565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d374e-de9b-425c-b33e-072d425ae700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:56.187189 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d374e-de9b-425c-b33e-072d425ae700 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:56.198782 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1281/2537] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:18:56.203272 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2a120f-9327-4f02-9777-ffd3234c516c admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:56.213353 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2a120f-9327-4f02-9777-ffd3234c516c admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:56.218518 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2aea6b-1b7d-40d7-bdec-365fc1060899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:56.219168 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2aea6b-1b7d-40d7-bdec-365fc1060899 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:56.219452 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2aea6b-1b7d-40d7-bdec-365fc1060899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:56.219728 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2aea6b-1b7d-40d7-bdec-365fc1060899 admin admin] PATH_INFO: `/v3/projects/544395ff430640d0a5b57e1fb201b47e/users/220a7e230dc84876bed3dc3ce8e16537/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:56.235061 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2aea6b-1b7d-40d7-bdec-365fc1060899 admin admin] RBAC: Authorizing `identity:create_grant(project_id=544395ff430640d0a5b57e1fb201b47e, user_id=220a7e230dc84876bed3dc3ce8e16537, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:56.237731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2aea6b-1b7d-40d7-bdec-365fc1060899 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:56.307436 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1257/2538] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:18:56 2022] PUT /identity/v3/projects/544395ff430640d0a5b57e1fb201b47e/users/220a7e230dc84876bed3dc3ce8e16537/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:18:56.317658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4097744-1a48-4653-8246-231d22fb7cd1 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:56.317991 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4097744-1a48-4653-8246-231d22fb7cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:56.318271 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4097744-1a48-4653-8246-231d22fb7cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:56.347411 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f4097744-1a48-4653-8246-231d22fb7cd1 None None] MFA Rules not processed for user `220a7e230dc84876bed3dc3ce8e16537`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:18:56.348638 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f4097744-1a48-4653-8246-231d22fb7cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:56.413288 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1282/2539] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:18:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:18:56.417907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2aea6b-1b7d-40d7-bdec-365fc1060899 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:56.483531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2aea6b-1b7d-40d7-bdec-365fc1060899 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:56.488347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70d2c9-099c-4157-96c9-e98f234e7678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:56.489133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70d2c9-099c-4157-96c9-e98f234e7678 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:56.489525 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70d2c9-099c-4157-96c9-e98f234e7678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:56.489696 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70d2c9-099c-4157-96c9-e98f234e7678 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:56.490413 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec70d2c9-099c-4157-96c9-e98f234e7678 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:56.492339 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec70d2c9-099c-4157-96c9-e98f234e7678 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:56.588850 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1258/2540] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:18:56 2022] POST /identity/v3/projects => generated 381 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:18:56.593727 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4097744-1a48-4653-8246-231d22fb7cd1 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:56.604742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4097744-1a48-4653-8246-231d22fb7cd1 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:56.663850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28430056-0617-45fb-84a7-175d59ee2edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:56.664477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28430056-0617-45fb-84a7-175d59ee2edf admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:56.664763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28430056-0617-45fb-84a7-175d59ee2edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:56.665043 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28430056-0617-45fb-84a7-175d59ee2edf admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:56.665746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28430056-0617-45fb-84a7-175d59ee2edf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:56.667719 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28430056-0617-45fb-84a7-175d59ee2edf admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:56.742757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1283/2541] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:18:56 2022] POST /identity/v3/users => generated 297 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:18:56.746778 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70d2c9-099c-4157-96c9-e98f234e7678 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:56.757429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70d2c9-099c-4157-96c9-e98f234e7678 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:56.764436 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2c685-12a4-4c24-a8d8-665908c64f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:56.764436 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2c685-12a4-4c24-a8d8-665908c64f5f admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:56.764436 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2c685-12a4-4c24-a8d8-665908c64f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:56.764436 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2c685-12a4-4c24-a8d8-665908c64f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:56.769355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a2c685-12a4-4c24-a8d8-665908c64f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:56.769355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a2c685-12a4-4c24-a8d8-665908c64f5f admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:56.772283 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:18:56.778528 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1259/2542] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:18:56.783192 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28430056-0617-45fb-84a7-175d59ee2edf admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:56.793843 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28430056-0617-45fb-84a7-175d59ee2edf admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:56.798746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b47f7d-1ece-4b37-90aa-e9965559f02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:56.799438 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b47f7d-1ece-4b37-90aa-e9965559f02d admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:56.799721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b47f7d-1ece-4b37-90aa-e9965559f02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:56.799998 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b47f7d-1ece-4b37-90aa-e9965559f02d admin admin] PATH_INFO: `/v3/projects/b27fb44e4bbd4ea69904edc897beedac/users/a7afc1a6640647b1bd8df53fb8fa4621/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:56.816504 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b47f7d-1ece-4b37-90aa-e9965559f02d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b27fb44e4bbd4ea69904edc897beedac, user_id=a7afc1a6640647b1bd8df53fb8fa4621, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:56.819428 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b47f7d-1ece-4b37-90aa-e9965559f02d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:56.886019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:18:56.891449 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1284/2543] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:18:56 2022] PUT /identity/v3/projects/b27fb44e4bbd4ea69904edc897beedac/users/a7afc1a6640647b1bd8df53fb8fa4621/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:18:56.895701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2c685-12a4-4c24-a8d8-665908c64f5f admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:56.906607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2c685-12a4-4c24-a8d8-665908c64f5f admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:56.965374 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2690fbcf-fde4-4fa4-b950-ae8b2206942c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:56.966030 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690fbcf-fde4-4fa4-b950-ae8b2206942c admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:56.966404 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690fbcf-fde4-4fa4-b950-ae8b2206942c admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:56.966682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2690fbcf-fde4-4fa4-b950-ae8b2206942c admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:56.967371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2690fbcf-fde4-4fa4-b950-ae8b2206942c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:56.968967 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2690fbcf-fde4-4fa4-b950-ae8b2206942c admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:56.978680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1260/2544] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:18:56.982963 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b47f7d-1ece-4b37-90aa-e9965559f02d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:56.993823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b47f7d-1ece-4b37-90aa-e9965559f02d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:56.999037 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a0d1ea-ab22-4c1c-86f9-59bc0c4570c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:56.999776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0d1ea-ab22-4c1c-86f9-59bc0c4570c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:57.000075 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0d1ea-ab22-4c1c-86f9-59bc0c4570c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:57.000380 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a0d1ea-ab22-4c1c-86f9-59bc0c4570c3 admin admin] PATH_INFO: `/v3/projects/b27fb44e4bbd4ea69904edc897beedac/users/a7afc1a6640647b1bd8df53fb8fa4621/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:57.002380 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a0d1ea-ab22-4c1c-86f9-59bc0c4570c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b27fb44e4bbd4ea69904edc897beedac, user_id=a7afc1a6640647b1bd8df53fb8fa4621, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:57.005032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a0d1ea-ab22-4c1c-86f9-59bc0c4570c3 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:57.015957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1285/2545] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:18:56 2022] PUT /identity/v3/projects/b27fb44e4bbd4ea69904edc897beedac/users/a7afc1a6640647b1bd8df53fb8fa4621/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:18:57.027975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce62ecd-4492-45b2-9bda-ca281d4dd0af None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:57.028285 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce62ecd-4492-45b2-9bda-ca281d4dd0af None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:57.028561 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce62ecd-4492-45b2-9bda-ca281d4dd0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:57.057228 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-4ce62ecd-4492-45b2-9bda-ca281d4dd0af None None] MFA Rules not processed for user `a7afc1a6640647b1bd8df53fb8fa4621`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:18:57.058458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4ce62ecd-4492-45b2-9bda-ca281d4dd0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:57.127325 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1261/2546] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:18:57 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:18:57.139228 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a0d1ea-ab22-4c1c-86f9-59bc0c4570c3 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:57.140901 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-87a0d1ea-ab22-4c1c-86f9-59bc0c4570c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:18:57.200895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a0d1ea-ab22-4c1c-86f9-59bc0c4570c3 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:57.204644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23382ffc-c48d-4249-b9a1-0522519b3bef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:57.205267 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23382ffc-c48d-4249-b9a1-0522519b3bef service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:57.205548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23382ffc-c48d-4249-b9a1-0522519b3bef service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:57.205823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23382ffc-c48d-4249-b9a1-0522519b3bef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:57.215088 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23382ffc-c48d-4249-b9a1-0522519b3bef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:57.216531 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23382ffc-c48d-4249-b9a1-0522519b3bef service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:57.226056 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1286/2547] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:18:57 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:18:57.510781 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce62ecd-4492-45b2-9bda-ca281d4dd0af None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:18:57.520735 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce62ecd-4492-45b2-9bda-ca281d4dd0af None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:18:57.524558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c98d1-ebdf-4427-a93b-a5c47c911fd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:18:57.525204 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c98d1-ebdf-4427-a93b-a5c47c911fd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:18:57.525485 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c98d1-ebdf-4427-a93b-a5c47c911fd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:18:57.525760 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c98d1-ebdf-4427-a93b-a5c47c911fd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:18:57.585731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c98d1-ebdf-4427-a93b-a5c47c911fd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:18:57.587252 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c98d1-ebdf-4427-a93b-a5c47c911fd3 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:18:57.596749 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1262/2548] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:18:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:19:03.395937 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23382ffc-c48d-4249-b9a1-0522519b3bef service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:03.397430 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-23382ffc-c48d-4249-b9a1-0522519b3bef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:03.463458 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23382ffc-c48d-4249-b9a1-0522519b3bef service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:03.468802 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca925b40-8670-4230-96cb-45f6211482eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:03.469450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca925b40-8670-4230-96cb-45f6211482eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:03.469750 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca925b40-8670-4230-96cb-45f6211482eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:03.470049 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca925b40-8670-4230-96cb-45f6211482eb admin admin] PATH_INFO: `/v3/users/d74394a975214a328cbd054465ec45aa` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:03.471215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca925b40-8670-4230-96cb-45f6211482eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=d74394a975214a328cbd054465ec45aa)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:03.473138 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca925b40-8670-4230-96cb-45f6211482eb admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:03.696883 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ca925b40-8670-4230-96cb-45f6211482eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd74394a975214a328cbd054465ec45aa'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:03.704615 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ca925b40-8670-4230-96cb-45f6211482eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd74394a975214a328cbd054465ec45aa'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:03.705286 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ca925b40-8670-4230-96cb-45f6211482eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd74394a975214a328cbd054465ec45aa'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:03.717424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ca925b40-8670-4230-96cb-45f6211482eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd74394a975214a328cbd054465ec45aa'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:03.718110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ca925b40-8670-4230-96cb-45f6211482eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd74394a975214a328cbd054465ec45aa'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:03.792190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1287/2549] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:19:03 2022] DELETE /identity/v3/users/d74394a975214a328cbd054465ec45aa => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:19:03.796731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028ca3d-008e-4026-b958-3c4f33175095 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:03.797250 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028ca3d-008e-4026-b958-3c4f33175095 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:03.797355 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f028ca3d-008e-4026-b958-3c4f33175095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:03.818258 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-f028ca3d-008e-4026-b958-3c4f33175095 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:03.819477 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f028ca3d-008e-4026-b958-3c4f33175095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:03.832957 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:03.877690 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1263/2550] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:03.887417 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca925b40-8670-4230-96cb-45f6211482eb admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:03.889399 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ca925b40-8670-4230-96cb-45f6211482eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:03.900114 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:03.949742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca925b40-8670-4230-96cb-45f6211482eb admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:03.952718 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e77d6c-e63d-468e-bcc3-a39b6b210793 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:03.953355 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e77d6c-e63d-468e-bcc3-a39b6b210793 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:03.953641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e77d6c-e63d-468e-bcc3-a39b6b210793 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:03.953947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e77d6c-e63d-468e-bcc3-a39b6b210793 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:03.963626 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e77d6c-e63d-468e-bcc3-a39b6b210793 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:03.965069 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e77d6c-e63d-468e-bcc3-a39b6b210793 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:03.975160 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1288/2551] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:19:04.393977 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f028ca3d-008e-4026-b958-3c4f33175095 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:04.395401 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f028ca3d-008e-4026-b958-3c4f33175095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:04.408182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f028ca3d-008e-4026-b958-3c4f33175095 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:04.412900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb6f9ce-f02a-4ddd-9604-64d28e4affca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:04.413599 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6f9ce-f02a-4ddd-9604-64d28e4affca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:04.413977 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6f9ce-f02a-4ddd-9604-64d28e4affca admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:04.414205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6f9ce-f02a-4ddd-9604-64d28e4affca admin admin] PATH_INFO: `/v3/projects/2a776c2f8e6b46578d1d9908947d2f66` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:04.415401 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb6f9ce-f02a-4ddd-9604-64d28e4affca admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a776c2f8e6b46578d1d9908947d2f66)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:04.419400 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb6f9ce-f02a-4ddd-9604-64d28e4affca admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:04.551789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-5fb6f9ce-f02a-4ddd-9604-64d28e4affca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a776c2f8e6b46578d1d9908947d2f66', 'request_id': 'req-5fb6f9ce-f02a-4ddd-9604-64d28e4affca'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:04.558921 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-5fb6f9ce-f02a-4ddd-9604-64d28e4affca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a776c2f8e6b46578d1d9908947d2f66', 'request_id': 'req-5fb6f9ce-f02a-4ddd-9604-64d28e4affca'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:04.565539 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-5fb6f9ce-f02a-4ddd-9604-64d28e4affca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a776c2f8e6b46578d1d9908947d2f66', 'request_id': 'req-5fb6f9ce-f02a-4ddd-9604-64d28e4affca'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:04.644229 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-5fb6f9ce-f02a-4ddd-9604-64d28e4affca admin admin] The token cache is being invalidate because project 2a776c2f8e6b46578d1d9908947d2f66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:19:04.644505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-5fb6f9ce-f02a-4ddd-9604-64d28e4affca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:04.645243 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-5fb6f9ce-f02a-4ddd-9604-64d28e4affca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:04.646859 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1264/2552] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:19:04 2022] DELETE /identity/v3/projects/2a776c2f8e6b46578d1d9908947d2f66 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:04.651719 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0b360-61f2-4fd1-bd8d-0c3e2475d02c None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:04.652222 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0b360-61f2-4fd1-bd8d-0c3e2475d02c None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:04.652342 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b0b360-61f2-4fd1-bd8d-0c3e2475d02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:04.672761 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-a6b0b360-61f2-4fd1-bd8d-0c3e2475d02c None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:04.674023 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a6b0b360-61f2-4fd1-bd8d-0c3e2475d02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:04.735174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1289/2553] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:19:04.743164 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508a1ca-bc3e-48e4-9546-e5115af991aa None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:04.743448 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508a1ca-bc3e-48e4-9546-e5115af991aa None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:04.743721 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508a1ca-bc3e-48e4-9546-e5115af991aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:04.764320 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-9508a1ca-bc3e-48e4-9546-e5115af991aa None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:04.765521 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-9508a1ca-bc3e-48e4-9546-e5115af991aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:04.773471 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1265/2554] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:04.777684 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0b360-61f2-4fd1-bd8d-0c3e2475d02c None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:04.787521 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b0b360-61f2-4fd1-bd8d-0c3e2475d02c None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:04.792117 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a3528-7662-486b-bf8e-30bafbece176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:04.792677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a3528-7662-486b-bf8e-30bafbece176 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:04.793035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a3528-7662-486b-bf8e-30bafbece176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:04.793308 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2a3528-7662-486b-bf8e-30bafbece176 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:04.794594 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a3528-7662-486b-bf8e-30bafbece176 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:19:04.794898 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a3528-7662-486b-bf8e-30bafbece176 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:04.796491 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2a3528-7662-486b-bf8e-30bafbece176 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:04.808641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1290/2555] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:19:04.813784 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508a1ca-bc3e-48e4-9546-e5115af991aa None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:04.823247 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508a1ca-bc3e-48e4-9546-e5115af991aa None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:04.827990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364406aa-d112-4fbe-a780-c3142df27f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:04.828599 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364406aa-d112-4fbe-a780-c3142df27f09 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:04.828881 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364406aa-d112-4fbe-a780-c3142df27f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:04.829154 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364406aa-d112-4fbe-a780-c3142df27f09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:04.829823 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364406aa-d112-4fbe-a780-c3142df27f09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:04.831759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364406aa-d112-4fbe-a780-c3142df27f09 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:04.913906 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1266/2556] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:19:04 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:19:04.924528 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a3528-7662-486b-bf8e-30bafbece176 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:04.935749 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2a3528-7662-486b-bf8e-30bafbece176 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:04.992164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d9383-c912-4499-8181-479a25bd5fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:04.992783 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d9383-c912-4499-8181-479a25bd5fa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:04.993066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d9383-c912-4499-8181-479a25bd5fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:04.993343 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d9383-c912-4499-8181-479a25bd5fa8 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:04.994054 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d9383-c912-4499-8181-479a25bd5fa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:04.996015 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d9383-c912-4499-8181-479a25bd5fa8 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:05.128756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1291/2557] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:19:04 2022] POST /identity/v3/users => generated 291 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:19:05.132884 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364406aa-d112-4fbe-a780-c3142df27f09 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:05.143138 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364406aa-d112-4fbe-a780-c3142df27f09 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:05.147657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9cdbe7-2f99-46fd-a105-5a8b73994869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:05.148259 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cdbe7-2f99-46fd-a105-5a8b73994869 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:05.148541 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cdbe7-2f99-46fd-a105-5a8b73994869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:05.148818 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cdbe7-2f99-46fd-a105-5a8b73994869 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:05.149505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9cdbe7-2f99-46fd-a105-5a8b73994869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:05.151144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9cdbe7-2f99-46fd-a105-5a8b73994869 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:05.162267 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1267/2558] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:19:05.166393 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d9383-c912-4499-8181-479a25bd5fa8 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:05.177602 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d9383-c912-4499-8181-479a25bd5fa8 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:05.182621 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b67a8-b357-4231-ba60-71c7673e0197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:05.183254 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b67a8-b357-4231-ba60-71c7673e0197 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:05.183535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b67a8-b357-4231-ba60-71c7673e0197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:05.183809 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b67a8-b357-4231-ba60-71c7673e0197 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:05.184532 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b67a8-b357-4231-ba60-71c7673e0197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:05.186152 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b67a8-b357-4231-ba60-71c7673e0197 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:05.197075 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1292/2559] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:19:05.201695 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9cdbe7-2f99-46fd-a105-5a8b73994869 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:05.211748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9cdbe7-2f99-46fd-a105-5a8b73994869 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:05.216984 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5e2de-38a2-49b4-be40-86724e0720f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:05.217620 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5e2de-38a2-49b4-be40-86724e0720f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:05.217924 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5e2de-38a2-49b4-be40-86724e0720f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:05.218233 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5e2de-38a2-49b4-be40-86724e0720f5 admin admin] PATH_INFO: `/v3/projects/473bba6b56ac4725b087e9fc1d55e9ef/users/235b4c823eed402fa50323629ce01723/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:05.233334 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5e2de-38a2-49b4-be40-86724e0720f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=473bba6b56ac4725b087e9fc1d55e9ef, user_id=235b4c823eed402fa50323629ce01723, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:05.236018 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5e2de-38a2-49b4-be40-86724e0720f5 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:05.334597 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1268/2560] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:05 2022] PUT /identity/v3/projects/473bba6b56ac4725b087e9fc1d55e9ef/users/235b4c823eed402fa50323629ce01723/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:05.345715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4784b68-cf65-4daa-bede-d2a8ef90e32b None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:05.346041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4784b68-cf65-4daa-bede-d2a8ef90e32b None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:05.346320 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4784b68-cf65-4daa-bede-d2a8ef90e32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:05.374217 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f4784b68-cf65-4daa-bede-d2a8ef90e32b None None] MFA Rules not processed for user `235b4c823eed402fa50323629ce01723`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:05.375428 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f4784b68-cf65-4daa-bede-d2a8ef90e32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:05.380911 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:05.440977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1293/2561] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:05.445608 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5e2de-38a2-49b4-be40-86724e0720f5 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:05.488806 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:05.507991 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5e2de-38a2-49b4-be40-86724e0720f5 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:05.513532 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984023c1-3334-40f4-858b-93898bc91f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:05.514193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984023c1-3334-40f4-858b-93898bc91f35 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:05.516536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984023c1-3334-40f4-858b-93898bc91f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:05.516536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984023c1-3334-40f4-858b-93898bc91f35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:05.516536 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984023c1-3334-40f4-858b-93898bc91f35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:05.517907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984023c1-3334-40f4-858b-93898bc91f35 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:05.605123 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1269/2562] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:19:05 2022] POST /identity/v3/projects => generated 379 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:19:05.609290 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4784b68-cf65-4daa-bede-d2a8ef90e32b None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:05.620030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4784b68-cf65-4daa-bede-d2a8ef90e32b None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:05.677361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db873948-29f0-4968-9c7a-64360654e5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:05.678100 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db873948-29f0-4968-9c7a-64360654e5f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:05.678280 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db873948-29f0-4968-9c7a-64360654e5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:05.678555 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db873948-29f0-4968-9c7a-64360654e5f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:05.679218 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db873948-29f0-4968-9c7a-64360654e5f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:05.681152 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db873948-29f0-4968-9c7a-64360654e5f3 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:05.758529 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1294/2563] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:19:05 2022] POST /identity/v3/users => generated 296 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:19:05.768440 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984023c1-3334-40f4-858b-93898bc91f35 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:05.780095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984023c1-3334-40f4-858b-93898bc91f35 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:05.784941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d680d0-9ae6-425d-8e54-d33db2f30d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:05.785558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d680d0-9ae6-425d-8e54-d33db2f30d45 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:05.785844 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d680d0-9ae6-425d-8e54-d33db2f30d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:05.786155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d680d0-9ae6-425d-8e54-d33db2f30d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:05.786847 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d680d0-9ae6-425d-8e54-d33db2f30d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:05.788572 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d680d0-9ae6-425d-8e54-d33db2f30d45 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:05.799707 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1270/2564] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:19:05.803686 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db873948-29f0-4968-9c7a-64360654e5f3 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:05.814859 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db873948-29f0-4968-9c7a-64360654e5f3 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:05.819517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e641e2-2a43-4a50-b44c-80b2f6cf1a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:05.820183 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e641e2-2a43-4a50-b44c-80b2f6cf1a32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:05.820466 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e641e2-2a43-4a50-b44c-80b2f6cf1a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:05.820741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e641e2-2a43-4a50-b44c-80b2f6cf1a32 admin admin] PATH_INFO: `/v3/projects/e85f86bd6f834576af5a60c4d329cdcd/users/fc778e04f2c8465e9ae69dd6ac6dbda0/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:05.836538 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e641e2-2a43-4a50-b44c-80b2f6cf1a32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e85f86bd6f834576af5a60c4d329cdcd, user_id=fc778e04f2c8465e9ae69dd6ac6dbda0, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:05.840143 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e641e2-2a43-4a50-b44c-80b2f6cf1a32 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:05.922843 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1295/2565] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:05 2022] PUT /identity/v3/projects/e85f86bd6f834576af5a60c4d329cdcd/users/fc778e04f2c8465e9ae69dd6ac6dbda0/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:19:05.927168 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d680d0-9ae6-425d-8e54-d33db2f30d45 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:05.938800 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d680d0-9ae6-425d-8e54-d33db2f30d45 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:05.996540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d706b40-f263-4ec7-a5fc-12bb4a994cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:05.997641 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d706b40-f263-4ec7-a5fc-12bb4a994cfc admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:05.997946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d706b40-f263-4ec7-a5fc-12bb4a994cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:05.998221 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d706b40-f263-4ec7-a5fc-12bb4a994cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:05.998931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d706b40-f263-4ec7-a5fc-12bb4a994cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:06.000489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d706b40-f263-4ec7-a5fc-12bb4a994cfc admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:06.010036 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1271/2566] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:19:06.014896 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e641e2-2a43-4a50-b44c-80b2f6cf1a32 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:06.025254 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e641e2-2a43-4a50-b44c-80b2f6cf1a32 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:06.029961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e95fc-c146-4479-8490-76523659d6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:06.030620 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e95fc-c146-4479-8490-76523659d6aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:06.030900 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e95fc-c146-4479-8490-76523659d6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:06.031173 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e95fc-c146-4479-8490-76523659d6aa admin admin] PATH_INFO: `/v3/projects/e85f86bd6f834576af5a60c4d329cdcd/users/fc778e04f2c8465e9ae69dd6ac6dbda0/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:06.033104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e95fc-c146-4479-8490-76523659d6aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e85f86bd6f834576af5a60c4d329cdcd, user_id=fc778e04f2c8465e9ae69dd6ac6dbda0, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:06.035816 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739e95fc-c146-4479-8490-76523659d6aa admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:06.045803 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1296/2567] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:06 2022] PUT /identity/v3/projects/e85f86bd6f834576af5a60c4d329cdcd/users/fc778e04f2c8465e9ae69dd6ac6dbda0/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:06.057726 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31dcb4f-931b-4bb0-b478-3f24602d40bf None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:06.058080 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31dcb4f-931b-4bb0-b478-3f24602d40bf None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:06.058433 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31dcb4f-931b-4bb0-b478-3f24602d40bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:06.087333 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-c31dcb4f-931b-4bb0-b478-3f24602d40bf None None] MFA Rules not processed for user `fc778e04f2c8465e9ae69dd6ac6dbda0`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:06.088642 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-c31dcb4f-931b-4bb0-b478-3f24602d40bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:06.160678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1272/2568] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:06.171591 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e95fc-c146-4479-8490-76523659d6aa admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:06.173288 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-739e95fc-c146-4479-8490-76523659d6aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:06.232052 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e95fc-c146-4479-8490-76523659d6aa admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:06.235776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e17aa-f9b2-44e0-a0e7-e80ebab66efa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:06.236397 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e17aa-f9b2-44e0-a0e7-e80ebab66efa service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:06.236680 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e17aa-f9b2-44e0-a0e7-e80ebab66efa service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:06.236957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7e17aa-f9b2-44e0-a0e7-e80ebab66efa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:06.246284 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e17aa-f9b2-44e0-a0e7-e80ebab66efa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:06.248329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7e17aa-f9b2-44e0-a0e7-e80ebab66efa service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:06.257951 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1297/2569] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:06 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:19:07.154054 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31dcb4f-931b-4bb0-b478-3f24602d40bf None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:07.163326 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31dcb4f-931b-4bb0-b478-3f24602d40bf None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:07.167973 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84305ce3-d517-4ed1-bbad-f3991ed780da tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc778e04f2c8465e9ae69dd6ac6dbda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e85f86bd6f834576af5a60c4d329cdcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:07.169311 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84305ce3-d517-4ed1-bbad-f3991ed780da tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:07.169425 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84305ce3-d517-4ed1-bbad-f3991ed780da tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:07.169622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84305ce3-d517-4ed1-bbad-f3991ed780da tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:07.170363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84305ce3-d517-4ed1-bbad-f3991ed780da tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:19:07.170671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84305ce3-d517-4ed1-bbad-f3991ed780da tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:07.172273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84305ce3-d517-4ed1-bbad-f3991ed780da tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:07.185395 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1273/2570] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:19:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:19:07.189688 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e17aa-f9b2-44e0-a0e7-e80ebab66efa service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:07.194126 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:07.200230 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7e17aa-f9b2-44e0-a0e7-e80ebab66efa service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:07.204819 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b04eec-05ea-4552-bb90-70699f8e00eb tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc778e04f2c8465e9ae69dd6ac6dbda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e85f86bd6f834576af5a60c4d329cdcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:07.205425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b04eec-05ea-4552-bb90-70699f8e00eb tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:07.205709 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b04eec-05ea-4552-bb90-70699f8e00eb tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:07.206037 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b04eec-05ea-4552-bb90-70699f8e00eb tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:07.206749 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b04eec-05ea-4552-bb90-70699f8e00eb tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:07.208656 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b04eec-05ea-4552-bb90-70699f8e00eb tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:07.312504 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1298/2571] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:19:07 2022] POST /identity/v3/projects => generated 353 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:19:07.316454 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84305ce3-d517-4ed1-bbad-f3991ed780da tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:07.327084 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84305ce3-d517-4ed1-bbad-f3991ed780da tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:07.346098 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:07.385020 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490717f8-ec15-400a-a7e9-962637b785de tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc778e04f2c8465e9ae69dd6ac6dbda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e85f86bd6f834576af5a60c4d329cdcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:07.385633 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490717f8-ec15-400a-a7e9-962637b785de tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:07.385943 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490717f8-ec15-400a-a7e9-962637b785de tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:07.386219 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490717f8-ec15-400a-a7e9-962637b785de tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:07.386906 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490717f8-ec15-400a-a7e9-962637b785de tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:19:07.387214 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490717f8-ec15-400a-a7e9-962637b785de tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:07.388795 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490717f8-ec15-400a-a7e9-962637b785de tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:07.400766 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1274/2572] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:19:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:19:07.442302 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b04eec-05ea-4552-bb90-70699f8e00eb tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:07.452825 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b04eec-05ea-4552-bb90-70699f8e00eb tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:07.457950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc807a-9a44-440b-a809-b99a497fd0a5 tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fc778e04f2c8465e9ae69dd6ac6dbda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e85f86bd6f834576af5a60c4d329cdcd', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:07.458352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc807a-9a44-440b-a809-b99a497fd0a5 tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:07.458621 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc807a-9a44-440b-a809-b99a497fd0a5 tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:07.458923 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc807a-9a44-440b-a809-b99a497fd0a5 tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] PATH_INFO: `/v3/projects/1d11665720d04327bb5b25d465fb44d1` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:07.460100 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc807a-9a44-440b-a809-b99a497fd0a5 tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1d11665720d04327bb5b25d465fb44d1)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:07.462020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc807a-9a44-440b-a809-b99a497fd0a5 tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:07.570493 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1acc807a-9a44-440b-a809-b99a497fd0a5 tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d11665720d04327bb5b25d465fb44d1', 'request_id': 'req-1acc807a-9a44-440b-a809-b99a497fd0a5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:07.576751 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1acc807a-9a44-440b-a809-b99a497fd0a5 tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d11665720d04327bb5b25d465fb44d1', 'request_id': 'req-1acc807a-9a44-440b-a809-b99a497fd0a5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:07.626774 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1acc807a-9a44-440b-a809-b99a497fd0a5 tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d11665720d04327bb5b25d465fb44d1', 'request_id': 'req-1acc807a-9a44-440b-a809-b99a497fd0a5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:07.639345 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1acc807a-9a44-440b-a809-b99a497fd0a5 tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] The token cache is being invalidate because project 1d11665720d04327bb5b25d465fb44d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:19:07.639635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1acc807a-9a44-440b-a809-b99a497fd0a5 tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:07.640313 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1acc807a-9a44-440b-a809-b99a497fd0a5 tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:07.642153 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1299/2573] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:19:07 2022] DELETE /identity/v3/projects/1d11665720d04327bb5b25d465fb44d1 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:08.228144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490717f8-ec15-400a-a7e9-962637b785de tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:08.230060 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-490717f8-ec15-400a-a7e9-962637b785de tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:08.298521 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490717f8-ec15-400a-a7e9-962637b785de tempest-VolumeQuotaClassesTest-328189892 tempest-VolumeQuotaClassesTest-328189892-project-admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:08.355055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c1c43-8fd1-453f-b771-6991bd01ca06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:08.355860 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c1c43-8fd1-453f-b771-6991bd01ca06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:08.356702 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c1c43-8fd1-453f-b771-6991bd01ca06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:08.356938 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318c1c43-8fd1-453f-b771-6991bd01ca06 admin admin] PATH_INFO: `/v3/users/235b4c823eed402fa50323629ce01723` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:08.358200 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c1c43-8fd1-453f-b771-6991bd01ca06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=235b4c823eed402fa50323629ce01723)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:08.360135 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318c1c43-8fd1-453f-b771-6991bd01ca06 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:08.612214 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-318c1c43-8fd1-453f-b771-6991bd01ca06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '235b4c823eed402fa50323629ce01723'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:08.619458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-318c1c43-8fd1-453f-b771-6991bd01ca06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '235b4c823eed402fa50323629ce01723'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:08.620155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-318c1c43-8fd1-453f-b771-6991bd01ca06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '235b4c823eed402fa50323629ce01723'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:08.712607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-318c1c43-8fd1-453f-b771-6991bd01ca06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '235b4c823eed402fa50323629ce01723'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:08.716143 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:08.728767 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-318c1c43-8fd1-453f-b771-6991bd01ca06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '235b4c823eed402fa50323629ce01723'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:08.735043 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1275/2574] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:19:08 2022] DELETE /identity/v3/users/235b4c823eed402fa50323629ce01723 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:08.739757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7331d3-04bf-4385-80fd-8bfa3b4c06c6 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:08.744170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7331d3-04bf-4385-80fd-8bfa3b4c06c6 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:08.744170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7331d3-04bf-4385-80fd-8bfa3b4c06c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:08.760531 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9f7331d3-04bf-4385-80fd-8bfa3b4c06c6 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:08.761968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9f7331d3-04bf-4385-80fd-8bfa3b4c06c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:08.823461 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1300/2575] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:19:08.834274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c1c43-8fd1-453f-b771-6991bd01ca06 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:08.835778 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-318c1c43-8fd1-453f-b771-6991bd01ca06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:08.897829 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318c1c43-8fd1-453f-b771-6991bd01ca06 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:08.901667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837af4f-1ed9-43e7-a421-5e196b2b646b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:08.902332 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837af4f-1ed9-43e7-a421-5e196b2b646b service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:08.902616 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837af4f-1ed9-43e7-a421-5e196b2b646b service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:08.902897 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837af4f-1ed9-43e7-a421-5e196b2b646b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:08.912515 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7837af4f-1ed9-43e7-a421-5e196b2b646b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:08.913985 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7837af4f-1ed9-43e7-a421-5e196b2b646b service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:08.923509 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1276/2576] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:19:09.398263 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7331d3-04bf-4385-80fd-8bfa3b4c06c6 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:09.399803 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9f7331d3-04bf-4385-80fd-8bfa3b4c06c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:09.413554 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7331d3-04bf-4385-80fd-8bfa3b4c06c6 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:09.419316 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d366372-083d-40d0-86a7-b54aca3a8b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:09.419973 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d366372-083d-40d0-86a7-b54aca3a8b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:09.420255 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d366372-083d-40d0-86a7-b54aca3a8b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:09.420534 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d366372-083d-40d0-86a7-b54aca3a8b07 admin admin] PATH_INFO: `/v3/projects/473bba6b56ac4725b087e9fc1d55e9ef` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:09.421722 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d366372-083d-40d0-86a7-b54aca3a8b07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=473bba6b56ac4725b087e9fc1d55e9ef)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:09.423717 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d366372-083d-40d0-86a7-b54aca3a8b07 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:09.581385 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6d366372-083d-40d0-86a7-b54aca3a8b07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '473bba6b56ac4725b087e9fc1d55e9ef', 'request_id': 'req-6d366372-083d-40d0-86a7-b54aca3a8b07'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:09.588396 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6d366372-083d-40d0-86a7-b54aca3a8b07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '473bba6b56ac4725b087e9fc1d55e9ef', 'request_id': 'req-6d366372-083d-40d0-86a7-b54aca3a8b07'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:09.593131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=85,keystone:INSERT=3,keystone:DELETE=12 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:09.665276 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6d366372-083d-40d0-86a7-b54aca3a8b07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '473bba6b56ac4725b087e9fc1d55e9ef', 'request_id': 'req-6d366372-083d-40d0-86a7-b54aca3a8b07'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:09.677975 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6d366372-083d-40d0-86a7-b54aca3a8b07 admin admin] The token cache is being invalidate because project 473bba6b56ac4725b087e9fc1d55e9ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:19:09.678278 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6d366372-083d-40d0-86a7-b54aca3a8b07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:09.678939 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6d366372-083d-40d0-86a7-b54aca3a8b07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:09.680464 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1301/2577] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:19:09 2022] DELETE /identity/v3/projects/473bba6b56ac4725b087e9fc1d55e9ef => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:09.684995 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837af4f-1ed9-43e7-a421-5e196b2b646b service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:09.702696 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837af4f-1ed9-43e7-a421-5e196b2b646b service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:09.763899 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6032694-bc18-4941-b438-2daecee13f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:09.764545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6032694-bc18-4941-b438-2daecee13f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:09.764830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6032694-bc18-4941-b438-2daecee13f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:09.765104 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6032694-bc18-4941-b438-2daecee13f3b admin admin] PATH_INFO: `/v3/users/fc778e04f2c8465e9ae69dd6ac6dbda0` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:09.766325 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6032694-bc18-4941-b438-2daecee13f3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc778e04f2c8465e9ae69dd6ac6dbda0)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:09.768460 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6032694-bc18-4941-b438-2daecee13f3b admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:09.968135 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-f6032694-bc18-4941-b438-2daecee13f3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc778e04f2c8465e9ae69dd6ac6dbda0'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:09.975625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-f6032694-bc18-4941-b438-2daecee13f3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc778e04f2c8465e9ae69dd6ac6dbda0'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:09.976454 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-f6032694-bc18-4941-b438-2daecee13f3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc778e04f2c8465e9ae69dd6ac6dbda0'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:10.044639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-f6032694-bc18-4941-b438-2daecee13f3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc778e04f2c8465e9ae69dd6ac6dbda0'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:10.057658 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-f6032694-bc18-4941-b438-2daecee13f3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc778e04f2c8465e9ae69dd6ac6dbda0'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:10.064465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1277/2578] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:19:09 2022] DELETE /identity/v3/users/fc778e04f2c8465e9ae69dd6ac6dbda0 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:10.494076 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d366372-083d-40d0-86a7-b54aca3a8b07 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:10.494076 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6d366372-083d-40d0-86a7-b54aca3a8b07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:10.505512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d366372-083d-40d0-86a7-b54aca3a8b07 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:10.565504 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692e1b3-30f0-4ca5-b018-f26221e79f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:10.566184 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692e1b3-30f0-4ca5-b018-f26221e79f19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:10.566470 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692e1b3-30f0-4ca5-b018-f26221e79f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:10.566750 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692e1b3-30f0-4ca5-b018-f26221e79f19 admin admin] PATH_INFO: `/v3/projects/e85f86bd6f834576af5a60c4d329cdcd` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:10.567920 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692e1b3-30f0-4ca5-b018-f26221e79f19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e85f86bd6f834576af5a60c4d329cdcd)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:10.570441 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692e1b3-30f0-4ca5-b018-f26221e79f19 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:10.655889 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9692e1b3-30f0-4ca5-b018-f26221e79f19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e85f86bd6f834576af5a60c4d329cdcd', 'request_id': 'req-9692e1b3-30f0-4ca5-b018-f26221e79f19'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:10.662120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9692e1b3-30f0-4ca5-b018-f26221e79f19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e85f86bd6f834576af5a60c4d329cdcd', 'request_id': 'req-9692e1b3-30f0-4ca5-b018-f26221e79f19'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:10.700963 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9692e1b3-30f0-4ca5-b018-f26221e79f19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e85f86bd6f834576af5a60c4d329cdcd', 'request_id': 'req-9692e1b3-30f0-4ca5-b018-f26221e79f19'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:10.711950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9692e1b3-30f0-4ca5-b018-f26221e79f19 admin admin] The token cache is being invalidate because project e85f86bd6f834576af5a60c4d329cdcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:19:10.712242 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9692e1b3-30f0-4ca5-b018-f26221e79f19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:10.712865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9692e1b3-30f0-4ca5-b018-f26221e79f19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:10.714460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1302/2579] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:19:10 2022] DELETE /identity/v3/projects/e85f86bd6f834576af5a60c4d329cdcd => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:10.719378 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df790c-9b87-446d-be21-cb085a2efd13 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:10.719894 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df790c-9b87-446d-be21-cb085a2efd13 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:10.719997 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df790c-9b87-446d-be21-cb085a2efd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:10.728804 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:10.740358 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-e5df790c-9b87-446d-be21-cb085a2efd13 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:10.741583 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e5df790c-9b87-446d-be21-cb085a2efd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:10.803884 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1278/2580] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:10.813138 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfc6ac-4a00-4259-ab77-17eabe49a024 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:10.813428 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfc6ac-4a00-4259-ab77-17eabe49a024 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:10.813734 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfc6ac-4a00-4259-ab77-17eabe49a024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:10.834430 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-11dfc6ac-4a00-4259-ab77-17eabe49a024 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:10.835650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-11dfc6ac-4a00-4259-ab77-17eabe49a024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:10.845365 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1303/2581] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:19:10.849722 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df790c-9b87-446d-be21-cb085a2efd13 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:10.861346 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df790c-9b87-446d-be21-cb085a2efd13 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:10.866232 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435899f4-0d8b-42f5-849e-d4ea58fdb956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:10.866877 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435899f4-0d8b-42f5-849e-d4ea58fdb956 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:10.867161 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435899f4-0d8b-42f5-849e-d4ea58fdb956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:10.867445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435899f4-0d8b-42f5-849e-d4ea58fdb956 admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:10.868147 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435899f4-0d8b-42f5-849e-d4ea58fdb956 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:19:10.868456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435899f4-0d8b-42f5-849e-d4ea58fdb956 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:10.870109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435899f4-0d8b-42f5-849e-d4ea58fdb956 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:10.882850 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1279/2582] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:19:10.887609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfc6ac-4a00-4259-ab77-17eabe49a024 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:10.897325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfc6ac-4a00-4259-ab77-17eabe49a024 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:10.902394 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c691a-69a6-4881-871f-ebf1aea9df3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:10.903033 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c691a-69a6-4881-871f-ebf1aea9df3c admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:10.903313 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c691a-69a6-4881-871f-ebf1aea9df3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:10.903587 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c691a-69a6-4881-871f-ebf1aea9df3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:10.904258 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c691a-69a6-4881-871f-ebf1aea9df3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:10.906236 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c691a-69a6-4881-871f-ebf1aea9df3c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:11.049203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1304/2583] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:19:10 2022] POST /identity/v3/projects => generated 387 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:19:11.050290 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435899f4-0d8b-42f5-849e-d4ea58fdb956 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:11.061206 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435899f4-0d8b-42f5-849e-d4ea58fdb956 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:11.118393 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bae937-30a5-4ad2-8ad8-62651de1e8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:11.119020 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bae937-30a5-4ad2-8ad8-62651de1e8d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:11.119306 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bae937-30a5-4ad2-8ad8-62651de1e8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:11.119584 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bae937-30a5-4ad2-8ad8-62651de1e8d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:11.120266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bae937-30a5-4ad2-8ad8-62651de1e8d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:11.122296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bae937-30a5-4ad2-8ad8-62651de1e8d6 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:11.225629 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1280/2584] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:19:11 2022] POST /identity/v3/users => generated 294 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:19:11.229759 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c691a-69a6-4881-871f-ebf1aea9df3c admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:11.241035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c691a-69a6-4881-871f-ebf1aea9df3c admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:11.245704 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f992f5-7aab-48c6-9d6d-214660f64289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:11.246435 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f992f5-7aab-48c6-9d6d-214660f64289 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:11.246713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f992f5-7aab-48c6-9d6d-214660f64289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:11.246995 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f992f5-7aab-48c6-9d6d-214660f64289 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:11.247695 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f992f5-7aab-48c6-9d6d-214660f64289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:11.249306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f992f5-7aab-48c6-9d6d-214660f64289 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:11.259833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1305/2585] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:19:11.263954 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bae937-30a5-4ad2-8ad8-62651de1e8d6 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:11.273964 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bae937-30a5-4ad2-8ad8-62651de1e8d6 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:11.283116 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d806f-2f19-418a-bf7b-0961b8c456d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:11.283116 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d806f-2f19-418a-bf7b-0961b8c456d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:11.283116 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d806f-2f19-418a-bf7b-0961b8c456d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:11.283116 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d806f-2f19-418a-bf7b-0961b8c456d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:11.283654 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d806f-2f19-418a-bf7b-0961b8c456d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:11.283654 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d806f-2f19-418a-bf7b-0961b8c456d0 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:11.295575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1281/2586] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:19:11.300047 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f992f5-7aab-48c6-9d6d-214660f64289 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:11.300884 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d806f-2f19-418a-bf7b-0961b8c456d0 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:11.303086 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-a90d806f-2f19-418a-bf7b-0961b8c456d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:11.311107 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f992f5-7aab-48c6-9d6d-214660f64289 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:11.315940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2615ec-6aa6-4f67-a880-b6492ca4414a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:11.316720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2615ec-6aa6-4f67-a880-b6492ca4414a admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:11.317118 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2615ec-6aa6-4f67-a880-b6492ca4414a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:11.317436 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2615ec-6aa6-4f67-a880-b6492ca4414a admin admin] PATH_INFO: `/v3/projects/ad2c8e89c9cc4621a1ea8671ad45996c/users/dc08522c51d64b179efed145764a5b2d/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:11.320438 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d806f-2f19-418a-bf7b-0961b8c456d0 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:11.326924 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72a5374-8e1b-41fc-a601-98bfe7b9593b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:11.327566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a5374-8e1b-41fc-a601-98bfe7b9593b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:11.327872 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a5374-8e1b-41fc-a601-98bfe7b9593b admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:11.328115 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72a5374-8e1b-41fc-a601-98bfe7b9593b admin admin] PATH_INFO: `/v3/users/220a7e230dc84876bed3dc3ce8e16537` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:11.329746 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a5374-8e1b-41fc-a601-98bfe7b9593b admin admin] RBAC: Authorizing `identity:delete_user(user_id=220a7e230dc84876bed3dc3ce8e16537)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:11.331825 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72a5374-8e1b-41fc-a601-98bfe7b9593b admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:11.336612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2615ec-6aa6-4f67-a880-b6492ca4414a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad2c8e89c9cc4621a1ea8671ad45996c, user_id=dc08522c51d64b179efed145764a5b2d, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:11.339315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2615ec-6aa6-4f67-a880-b6492ca4414a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:11.432996 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1306/2587] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:11 2022] PUT /identity/v3/projects/ad2c8e89c9cc4621a1ea8671ad45996c/users/dc08522c51d64b179efed145764a5b2d/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:11.443434 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f2bd2-fb75-4265-951f-100b5f6db55d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:11.443725 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f2bd2-fb75-4265-951f-100b5f6db55d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:11.444006 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f2bd2-fb75-4265-951f-100b5f6db55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:11.469966 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:11.472967 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-715f2bd2-fb75-4265-951f-100b5f6db55d None None] MFA Rules not processed for user `dc08522c51d64b179efed145764a5b2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:11.474220 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-715f2bd2-fb75-4265-951f-100b5f6db55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:11.532986 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-a72a5374-8e1b-41fc-a601-98bfe7b9593b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '220a7e230dc84876bed3dc3ce8e16537'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:11.538627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1307/2588] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:19:11.543035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-a72a5374-8e1b-41fc-a601-98bfe7b9593b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '220a7e230dc84876bed3dc3ce8e16537'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:11.543205 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f2bd2-fb75-4265-951f-100b5f6db55d None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:11.543740 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-a72a5374-8e1b-41fc-a601-98bfe7b9593b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '220a7e230dc84876bed3dc3ce8e16537'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:11.604895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f2bd2-fb75-4265-951f-100b5f6db55d None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:11.610767 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480432e-1147-4ef6-ad70-9fb68583c57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:11.611429 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480432e-1147-4ef6-ad70-9fb68583c57b admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:11.611710 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480432e-1147-4ef6-ad70-9fb68583c57b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:11.611985 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480432e-1147-4ef6-ad70-9fb68583c57b admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:11.612647 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480432e-1147-4ef6-ad70-9fb68583c57b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:11.614594 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8480432e-1147-4ef6-ad70-9fb68583c57b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:11.640155 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-a72a5374-8e1b-41fc-a601-98bfe7b9593b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '220a7e230dc84876bed3dc3ce8e16537'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:11.642558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:11.653852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-a72a5374-8e1b-41fc-a601-98bfe7b9593b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '220a7e230dc84876bed3dc3ce8e16537'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:11.661614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1282/2589] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:19:11 2022] DELETE /identity/v3/users/220a7e230dc84876bed3dc3ce8e16537 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:11.666416 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b7c82-6cb7-483e-845f-77307e97abcd None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:11.668111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b7c82-6cb7-483e-845f-77307e97abcd None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:11.668111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b7c82-6cb7-483e-845f-77307e97abcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:11.689225 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-376b7c82-6cb7-483e-845f-77307e97abcd None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:11.693770 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-376b7c82-6cb7-483e-845f-77307e97abcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:11.724019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1308/2590] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:19:11 2022] POST /identity/v3/projects => generated 389 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:19:11.725104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480432e-1147-4ef6-ad70-9fb68583c57b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:11.726889 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8480432e-1147-4ef6-ad70-9fb68583c57b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:11.754186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1283/2591] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:11.770778 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b7c82-6cb7-483e-845f-77307e97abcd None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:11.772183 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-376b7c82-6cb7-483e-845f-77307e97abcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:11.793226 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8480432e-1147-4ef6-ad70-9fb68583c57b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:11.798241 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e5208-f059-4be4-8dc5-695ece75c4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:11.798866 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e5208-f059-4be4-8dc5-695ece75c4db admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:11.799148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e5208-f059-4be4-8dc5-695ece75c4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:11.799423 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e5208-f059-4be4-8dc5-695ece75c4db admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:11.800098 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e5208-f059-4be4-8dc5-695ece75c4db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:11.802067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e5208-f059-4be4-8dc5-695ece75c4db admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:11.832867 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b7c82-6cb7-483e-845f-77307e97abcd None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:11.879633 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f587b0-ba58-4a96-bf89-2af95fb645b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:11.880244 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f587b0-ba58-4a96-bf89-2af95fb645b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:11.880524 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f587b0-ba58-4a96-bf89-2af95fb645b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:11.880799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f587b0-ba58-4a96-bf89-2af95fb645b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:11.890562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f587b0-ba58-4a96-bf89-2af95fb645b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:11.891993 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f587b0-ba58-4a96-bf89-2af95fb645b8 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:11.901547 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1284/2592] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:19:11.907066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1309/2593] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:19:11 2022] POST /identity/v3/users => generated 301 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:19:11.910705 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f587b0-ba58-4a96-bf89-2af95fb645b8 service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:11.921052 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f587b0-ba58-4a96-bf89-2af95fb645b8 service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:11.929595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6ca11-ef24-4bf0-a562-fceaa266a9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:11.930282 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6ca11-ef24-4bf0-a562-fceaa266a9bc admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:11.930572 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6ca11-ef24-4bf0-a562-fceaa266a9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:11.930852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6ca11-ef24-4bf0-a562-fceaa266a9bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:11.931809 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6ca11-ef24-4bf0-a562-fceaa266a9bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:11.933452 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6ca11-ef24-4bf0-a562-fceaa266a9bc admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:11.955019 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1285/2594] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:19:11.956704 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e5208-f059-4be4-8dc5-695ece75c4db admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:11.967988 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e5208-f059-4be4-8dc5-695ece75c4db admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:11.972888 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4196aa6-c90d-4eae-87f5-b844a660c4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:11.973576 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4196aa6-c90d-4eae-87f5-b844a660c4d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:11.973863 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4196aa6-c90d-4eae-87f5-b844a660c4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:11.974176 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4196aa6-c90d-4eae-87f5-b844a660c4d8 admin admin] PATH_INFO: `/v3/projects/58d7b7da252048efae4dff8a35d92dd7/users/6a4d3dd92d0844ef8e2e05af14d16a10/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:11.990462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4196aa6-c90d-4eae-87f5-b844a660c4d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58d7b7da252048efae4dff8a35d92dd7, user_id=6a4d3dd92d0844ef8e2e05af14d16a10, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:11.993252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4196aa6-c90d-4eae-87f5-b844a660c4d8 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:11.995992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:12.107059 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1310/2595] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:11 2022] PUT /identity/v3/projects/58d7b7da252048efae4dff8a35d92dd7/users/6a4d3dd92d0844ef8e2e05af14d16a10/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:12.111035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6ca11-ef24-4bf0-a562-fceaa266a9bc admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:12.122360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6ca11-ef24-4bf0-a562-fceaa266a9bc admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:12.157079 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:12.186893 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3e4480-285c-4fa3-943f-d486647571ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:12.191049 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e4480-285c-4fa3-943f-d486647571ba admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:12.191049 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e4480-285c-4fa3-943f-d486647571ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:12.191049 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3e4480-285c-4fa3-943f-d486647571ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:12.191049 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e4480-285c-4fa3-943f-d486647571ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:12.191049 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3e4480-285c-4fa3-943f-d486647571ba admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:12.200853 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1286/2596] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:19:12.205033 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4196aa6-c90d-4eae-87f5-b844a660c4d8 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:12.216541 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4196aa6-c90d-4eae-87f5-b844a660c4d8 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:12.221352 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb7539-46b1-4f29-a743-539234a13bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:12.222063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb7539-46b1-4f29-a743-539234a13bfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:12.222350 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb7539-46b1-4f29-a743-539234a13bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:12.222661 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb7539-46b1-4f29-a743-539234a13bfa admin admin] PATH_INFO: `/v3/projects/58d7b7da252048efae4dff8a35d92dd7/users/6a4d3dd92d0844ef8e2e05af14d16a10/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:12.224657 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb7539-46b1-4f29-a743-539234a13bfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=58d7b7da252048efae4dff8a35d92dd7, user_id=6a4d3dd92d0844ef8e2e05af14d16a10, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:12.227365 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb7539-46b1-4f29-a743-539234a13bfa admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:12.240618 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1311/2597] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:12 2022] PUT /identity/v3/projects/58d7b7da252048efae4dff8a35d92dd7/users/6a4d3dd92d0844ef8e2e05af14d16a10/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:12.252632 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7be3f1-360f-4893-8b21-a4c6d4eef044 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:12.252924 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7be3f1-360f-4893-8b21-a4c6d4eef044 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:12.253197 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7be3f1-360f-4893-8b21-a4c6d4eef044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:12.282645 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-be7be3f1-360f-4893-8b21-a4c6d4eef044 None None] MFA Rules not processed for user `6a4d3dd92d0844ef8e2e05af14d16a10`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:12.283857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-be7be3f1-360f-4893-8b21-a4c6d4eef044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:12.361328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1287/2598] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:12 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:19:12.362528 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb7539-46b1-4f29-a743-539234a13bfa admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:12.364646 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-28fb7539-46b1-4f29-a743-539234a13bfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:12.370002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7be3f1-360f-4893-8b21-a4c6d4eef044 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:12.371454 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-be7be3f1-360f-4893-8b21-a4c6d4eef044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:12.433429 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7be3f1-360f-4893-8b21-a4c6d4eef044 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:12.437283 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd9d6f-6f55-4f65-b816-350aa47b26b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:12.437956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd9d6f-6f55-4f65-b816-350aa47b26b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:12.438243 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd9d6f-6f55-4f65-b816-350aa47b26b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:12.438521 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bd9d6f-6f55-4f65-b816-350aa47b26b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:12.448339 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd9d6f-6f55-4f65-b816-350aa47b26b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:12.449065 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb7539-46b1-4f29-a743-539234a13bfa admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:12.450722 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bd9d6f-6f55-4f65-b816-350aa47b26b8 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:12.454424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d319dde-d1df-49ca-b2e8-6256f61429f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:12.455074 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d319dde-d1df-49ca-b2e8-6256f61429f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:12.455483 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d319dde-d1df-49ca-b2e8-6256f61429f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:12.455767 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d319dde-d1df-49ca-b2e8-6256f61429f8 admin admin] PATH_INFO: `/v3/projects/544395ff430640d0a5b57e1fb201b47e` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:12.456950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d319dde-d1df-49ca-b2e8-6256f61429f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=544395ff430640d0a5b57e1fb201b47e)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:12.458919 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d319dde-d1df-49ca-b2e8-6256f61429f8 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:12.461105 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1288/2599] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:12 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:19:12.577793 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4d319dde-d1df-49ca-b2e8-6256f61429f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '544395ff430640d0a5b57e1fb201b47e', 'request_id': 'req-4d319dde-d1df-49ca-b2e8-6256f61429f8'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:12.584305 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4d319dde-d1df-49ca-b2e8-6256f61429f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '544395ff430640d0a5b57e1fb201b47e', 'request_id': 'req-4d319dde-d1df-49ca-b2e8-6256f61429f8'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:12.599577 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd9d6f-6f55-4f65-b816-350aa47b26b8 service cinder] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:12.610823 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bd9d6f-6f55-4f65-b816-350aa47b26b8 service cinder] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:12.661654 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:12.664751 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4d319dde-d1df-49ca-b2e8-6256f61429f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '544395ff430640d0a5b57e1fb201b47e', 'request_id': 'req-4d319dde-d1df-49ca-b2e8-6256f61429f8'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:12.669010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d9922-1dbd-4e2b-846a-6cfec8bf03a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:12.669660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d9922-1dbd-4e2b-846a-6cfec8bf03a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:12.669974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d9922-1dbd-4e2b-846a-6cfec8bf03a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:12.670259 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d9922-1dbd-4e2b-846a-6cfec8bf03a6 admin admin] PATH_INFO: `/v3/users/dc08522c51d64b179efed145764a5b2d` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:12.674761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d9922-1dbd-4e2b-846a-6cfec8bf03a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc08522c51d64b179efed145764a5b2d)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:12.674761 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d9922-1dbd-4e2b-846a-6cfec8bf03a6 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:12.678127 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4d319dde-d1df-49ca-b2e8-6256f61429f8 admin admin] The token cache is being invalidate because project 544395ff430640d0a5b57e1fb201b47e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:19:12.678424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4d319dde-d1df-49ca-b2e8-6256f61429f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:12.679100 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4d319dde-d1df-49ca-b2e8-6256f61429f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:12.680754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1312/2600] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:19:12 2022] DELETE /identity/v3/projects/544395ff430640d0a5b57e1fb201b47e => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:19:12.685358 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d319dde-d1df-49ca-b2e8-6256f61429f8 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:12.686863 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4d319dde-d1df-49ca-b2e8-6256f61429f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:12.700444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d319dde-d1df-49ca-b2e8-6256f61429f8 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:12.705124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423a0c9-fd53-46e7-b82a-5c1914de1bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:12.705768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423a0c9-fd53-46e7-b82a-5c1914de1bd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:12.706076 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423a0c9-fd53-46e7-b82a-5c1914de1bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:12.706351 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423a0c9-fd53-46e7-b82a-5c1914de1bd1 admin admin] PATH_INFO: `/v3/users/a7afc1a6640647b1bd8df53fb8fa4621` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:12.707523 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423a0c9-fd53-46e7-b82a-5c1914de1bd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7afc1a6640647b1bd8df53fb8fa4621)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:12.710067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423a0c9-fd53-46e7-b82a-5c1914de1bd1 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:12.857870 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-d86d9922-1dbd-4e2b-846a-6cfec8bf03a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc08522c51d64b179efed145764a5b2d'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:12.864836 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-d86d9922-1dbd-4e2b-846a-6cfec8bf03a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc08522c51d64b179efed145764a5b2d'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:12.865474 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-d86d9922-1dbd-4e2b-846a-6cfec8bf03a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc08522c51d64b179efed145764a5b2d'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:12.944043 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-7423a0c9-fd53-46e7-b82a-5c1914de1bd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7afc1a6640647b1bd8df53fb8fa4621'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:12.951310 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-7423a0c9-fd53-46e7-b82a-5c1914de1bd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7afc1a6640647b1bd8df53fb8fa4621'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:12.952009 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-7423a0c9-fd53-46e7-b82a-5c1914de1bd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7afc1a6640647b1bd8df53fb8fa4621'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:12.959076 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:INSERT=3,keystone:SELECT=82,keystone:DELETE=15 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:12.963306 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-d86d9922-1dbd-4e2b-846a-6cfec8bf03a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc08522c51d64b179efed145764a5b2d'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:12.964609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-7423a0c9-fd53-46e7-b82a-5c1914de1bd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7afc1a6640647b1bd8df53fb8fa4621'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:12.965395 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-7423a0c9-fd53-46e7-b82a-5c1914de1bd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7afc1a6640647b1bd8df53fb8fa4621'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:12.976575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-d86d9922-1dbd-4e2b-846a-6cfec8bf03a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc08522c51d64b179efed145764a5b2d'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:12.983181 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1289/2601] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:19:12 2022] DELETE /identity/v3/users/dc08522c51d64b179efed145764a5b2d => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:19:12.987431 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b4c53-d157-4363-9dea-96bfb14b3eea None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:12.987724 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b4c53-d157-4363-9dea-96bfb14b3eea None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:12.988008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b4c53-d157-4363-9dea-96bfb14b3eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:13.007375 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-f67b4c53-d157-4363-9dea-96bfb14b3eea None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:13.008624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f67b4c53-d157-4363-9dea-96bfb14b3eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:13.055292 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1313/2602] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:19:12 2022] DELETE /identity/v3/users/a7afc1a6640647b1bd8df53fb8fa4621 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:13.070380 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1290/2603] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:19:13.080893 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423a0c9-fd53-46e7-b82a-5c1914de1bd1 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:13.082606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7423a0c9-fd53-46e7-b82a-5c1914de1bd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:13.144160 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423a0c9-fd53-46e7-b82a-5c1914de1bd1 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:13.147742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759135c-b894-48ff-afb8-7bdd389d8338 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:13.148375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759135c-b894-48ff-afb8-7bdd389d8338 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:13.148658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759135c-b894-48ff-afb8-7bdd389d8338 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:13.148935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759135c-b894-48ff-afb8-7bdd389d8338 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:13.210050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759135c-b894-48ff-afb8-7bdd389d8338 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:13.211494 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759135c-b894-48ff-afb8-7bdd389d8338 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:13.220979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1314/2604] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:19:13.531123 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b4c53-d157-4363-9dea-96bfb14b3eea None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:13.532575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f67b4c53-d157-4363-9dea-96bfb14b3eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:13.545147 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b4c53-d157-4363-9dea-96bfb14b3eea None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:13.550861 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b0974a-a3c9-4956-888f-37adf71371b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:13.551522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0974a-a3c9-4956-888f-37adf71371b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:13.551803 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0974a-a3c9-4956-888f-37adf71371b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:13.552083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b0974a-a3c9-4956-888f-37adf71371b5 admin admin] PATH_INFO: `/v3/projects/b27fb44e4bbd4ea69904edc897beedac` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:13.553421 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0974a-a3c9-4956-888f-37adf71371b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b27fb44e4bbd4ea69904edc897beedac)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:13.555755 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b0974a-a3c9-4956-888f-37adf71371b5 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:13.679765 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759135c-b894-48ff-afb8-7bdd389d8338 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:13.680869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b3b0974a-a3c9-4956-888f-37adf71371b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b27fb44e4bbd4ea69904edc897beedac', 'request_id': 'req-b3b0974a-a3c9-4956-888f-37adf71371b5'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:13.681763 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3759135c-b894-48ff-afb8-7bdd389d8338 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:13.689422 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b3b0974a-a3c9-4956-888f-37adf71371b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b27fb44e4bbd4ea69904edc897beedac', 'request_id': 'req-b3b0974a-a3c9-4956-888f-37adf71371b5'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:13.696396 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759135c-b894-48ff-afb8-7bdd389d8338 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:13.696575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b3b0974a-a3c9-4956-888f-37adf71371b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b27fb44e4bbd4ea69904edc897beedac', 'request_id': 'req-b3b0974a-a3c9-4956-888f-37adf71371b5'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:13.753267 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:13.754403 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5521c9de-c2f4-4d55-80da-2871fd843f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:13.755057 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521c9de-c2f4-4d55-80da-2871fd843f9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:13.755341 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521c9de-c2f4-4d55-80da-2871fd843f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:13.755620 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521c9de-c2f4-4d55-80da-2871fd843f9d admin admin] PATH_INFO: `/v3/projects/ad2c8e89c9cc4621a1ea8671ad45996c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:13.756833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5521c9de-c2f4-4d55-80da-2871fd843f9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad2c8e89c9cc4621a1ea8671ad45996c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:13.758028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b3b0974a-a3c9-4956-888f-37adf71371b5 admin admin] The token cache is being invalidate because project b27fb44e4bbd4ea69904edc897beedac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:19:13.758326 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b3b0974a-a3c9-4956-888f-37adf71371b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:13.758810 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5521c9de-c2f4-4d55-80da-2871fd843f9d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:13.758956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-b3b0974a-a3c9-4956-888f-37adf71371b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:13.762223 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1291/2605] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:19:13 2022] DELETE /identity/v3/projects/b27fb44e4bbd4ea69904edc897beedac => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:13.765262 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd381da-1938-47e7-9f32-a425455bb035 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:13.765561 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd381da-1938-47e7-9f32-a425455bb035 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:13.765843 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd381da-1938-47e7-9f32-a425455bb035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:13.786485 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-bcd381da-1938-47e7-9f32-a425455bb035 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:13.787716 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-bcd381da-1938-47e7-9f32-a425455bb035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:13.796699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1292/2606] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:19:13.805019 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24426bef-e9de-446d-805f-088838ffad55 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:13.805304 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24426bef-e9de-446d-805f-088838ffad55 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:13.805579 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24426bef-e9de-446d-805f-088838ffad55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:13.826031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-24426bef-e9de-446d-805f-088838ffad55 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:13.827242 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-24426bef-e9de-446d-805f-088838ffad55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:13.835397 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1293/2607] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:19:13.840304 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24426bef-e9de-446d-805f-088838ffad55 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:13.848615 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24426bef-e9de-446d-805f-088838ffad55 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:13.853269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32640e-cb6d-43d3-a992-7d5d92f82b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:13.853864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32640e-cb6d-43d3-a992-7d5d92f82b3a admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:13.854177 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32640e-cb6d-43d3-a992-7d5d92f82b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:13.854451 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc32640e-cb6d-43d3-a992-7d5d92f82b3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:13.855140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32640e-cb6d-43d3-a992-7d5d92f82b3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:19:13.855446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32640e-cb6d-43d3-a992-7d5d92f82b3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:13.857571 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc32640e-cb6d-43d3-a992-7d5d92f82b3a admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:13.868198 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:13.869352 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1294/2608] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:19:13.874306 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32640e-cb6d-43d3-a992-7d5d92f82b3a admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:13.882074 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5521c9de-c2f4-4d55-80da-2871fd843f9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad2c8e89c9cc4621a1ea8671ad45996c', 'request_id': 'req-5521c9de-c2f4-4d55-80da-2871fd843f9d'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:13.884386 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc32640e-cb6d-43d3-a992-7d5d92f82b3a admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:13.888827 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5521c9de-c2f4-4d55-80da-2871fd843f9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad2c8e89c9cc4621a1ea8671ad45996c', 'request_id': 'req-5521c9de-c2f4-4d55-80da-2871fd843f9d'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:13.942634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89765ed-e330-4a66-b913-a27ec0ecbf9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:13.943266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89765ed-e330-4a66-b913-a27ec0ecbf9e admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:13.943552 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89765ed-e330-4a66-b913-a27ec0ecbf9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:13.943832 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89765ed-e330-4a66-b913-a27ec0ecbf9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:13.944505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89765ed-e330-4a66-b913-a27ec0ecbf9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:13.946453 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89765ed-e330-4a66-b913-a27ec0ecbf9e admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:13.990467 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5521c9de-c2f4-4d55-80da-2871fd843f9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad2c8e89c9cc4621a1ea8671ad45996c', 'request_id': 'req-5521c9de-c2f4-4d55-80da-2871fd843f9d'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:14.002435 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5521c9de-c2f4-4d55-80da-2871fd843f9d admin admin] The token cache is being invalidate because project ad2c8e89c9cc4621a1ea8671ad45996c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:19:14.002748 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5521c9de-c2f4-4d55-80da-2871fd843f9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:14.003387 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-5521c9de-c2f4-4d55-80da-2871fd843f9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:14.004918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1315/2609] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:19:13 2022] DELETE /identity/v3/projects/ad2c8e89c9cc4621a1ea8671ad45996c => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:14.008918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5521c9de-c2f4-4d55-80da-2871fd843f9d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:14.010258 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5521c9de-c2f4-4d55-80da-2871fd843f9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:14.022596 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5521c9de-c2f4-4d55-80da-2871fd843f9d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:14.027216 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822f05a-c55d-4a5b-b3b4-15cc232802e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:14.028355 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822f05a-c55d-4a5b-b3b4-15cc232802e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:14.028627 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822f05a-c55d-4a5b-b3b4-15cc232802e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:14.028901 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822f05a-c55d-4a5b-b3b4-15cc232802e2 admin admin] PATH_INFO: `/v3/users/6a4d3dd92d0844ef8e2e05af14d16a10` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:14.030106 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f822f05a-c55d-4a5b-b3b4-15cc232802e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a4d3dd92d0844ef8e2e05af14d16a10)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:14.032097 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f822f05a-c55d-4a5b-b3b4-15cc232802e2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:14.039103 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1295/2610] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:19:13 2022] POST /identity/v3/projects => generated 385 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:19:14.043742 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89765ed-e330-4a66-b913-a27ec0ecbf9e admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:14.046182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-b89765ed-e330-4a66-b913-a27ec0ecbf9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:14.110759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89765ed-e330-4a66-b913-a27ec0ecbf9e admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:14.115580 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73feae84-f1b4-490f-856a-d0a33178f035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:14.116203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73feae84-f1b4-490f-856a-d0a33178f035 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:14.116480 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73feae84-f1b4-490f-856a-d0a33178f035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:14.116754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73feae84-f1b4-490f-856a-d0a33178f035 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:14.117421 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73feae84-f1b4-490f-856a-d0a33178f035 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:14.119368 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73feae84-f1b4-490f-856a-d0a33178f035 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:14.171430 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f822f05a-c55d-4a5b-b3b4-15cc232802e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a4d3dd92d0844ef8e2e05af14d16a10'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:14.178314 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f822f05a-c55d-4a5b-b3b4-15cc232802e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a4d3dd92d0844ef8e2e05af14d16a10'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:14.179036 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f822f05a-c55d-4a5b-b3b4-15cc232802e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a4d3dd92d0844ef8e2e05af14d16a10'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:14.191507 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f822f05a-c55d-4a5b-b3b4-15cc232802e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a4d3dd92d0844ef8e2e05af14d16a10'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:14.192101 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f822f05a-c55d-4a5b-b3b4-15cc232802e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a4d3dd92d0844ef8e2e05af14d16a10'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:14.204556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1296/2611] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:19:14 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:19:14.208494 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73feae84-f1b4-490f-856a-d0a33178f035 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:14.210126 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-73feae84-f1b4-490f-856a-d0a33178f035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:14.222171 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73feae84-f1b4-490f-856a-d0a33178f035 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:14.226759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd30db-88e6-4b1f-8536-6f7b3d1a5c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:14.227356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd30db-88e6-4b1f-8536-6f7b3d1a5c87 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:14.227635 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd30db-88e6-4b1f-8536-6f7b3d1a5c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:14.227919 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd30db-88e6-4b1f-8536-6f7b3d1a5c87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:14.228582 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd30db-88e6-4b1f-8536-6f7b3d1a5c87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:14.230623 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcd30db-88e6-4b1f-8536-6f7b3d1a5c87 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:14.240403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1297/2612] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:19:14.244966 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1316/2613] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:19:14 2022] DELETE /identity/v3/users/6a4d3dd92d0844ef8e2e05af14d16a10 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:19:14.249290 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd30db-88e6-4b1f-8536-6f7b3d1a5c87 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:14.259907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcd30db-88e6-4b1f-8536-6f7b3d1a5c87 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:14.319744 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf087bd6-16c2-498a-b656-41b2a32d125c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:14.320351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf087bd6-16c2-498a-b656-41b2a32d125c admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:14.320632 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf087bd6-16c2-498a-b656-41b2a32d125c admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:14.320911 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf087bd6-16c2-498a-b656-41b2a32d125c admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:14.321643 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf087bd6-16c2-498a-b656-41b2a32d125c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:14.323298 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf087bd6-16c2-498a-b656-41b2a32d125c admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:14.337810 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1298/2614] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:19:14.338976 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822f05a-c55d-4a5b-b3b4-15cc232802e2 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:14.349090 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822f05a-c55d-4a5b-b3b4-15cc232802e2 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:14.354139 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb55a9-6e4d-452c-b652-5d8cd606ac83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:14.355010 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb55a9-6e4d-452c-b652-5d8cd606ac83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:14.359588 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb55a9-6e4d-452c-b652-5d8cd606ac83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:14.359588 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb55a9-6e4d-452c-b652-5d8cd606ac83 admin admin] PATH_INFO: `/v3/projects/3901f7fcbb76480e8703b4898a09500d/users/28a9891a6857409b971e526c91bb2e72/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:14.370571 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb55a9-6e4d-452c-b652-5d8cd606ac83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3901f7fcbb76480e8703b4898a09500d, user_id=28a9891a6857409b971e526c91bb2e72, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:14.373740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb55a9-6e4d-452c-b652-5d8cd606ac83 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:14.438283 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:14.446095 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1317/2615] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:14 2022] PUT /identity/v3/projects/3901f7fcbb76480e8703b4898a09500d/users/28a9891a6857409b971e526c91bb2e72/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:19:14.456997 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bca5a1-ad35-4929-a018-dd8b45f1c249 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:14.457296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bca5a1-ad35-4929-a018-dd8b45f1c249 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:14.457581 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bca5a1-ad35-4929-a018-dd8b45f1c249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:14.488408 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-84bca5a1-ad35-4929-a018-dd8b45f1c249 None None] MFA Rules not processed for user `28a9891a6857409b971e526c91bb2e72`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:14.489647 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-84bca5a1-ad35-4929-a018-dd8b45f1c249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:14.555747 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1299/2616] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:14.561362 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb55a9-6e4d-452c-b652-5d8cd606ac83 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:14.626577 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb55a9-6e4d-452c-b652-5d8cd606ac83 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:14.631295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b771947-86df-4e4d-af83-56644d623995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:14.631916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b771947-86df-4e4d-af83-56644d623995 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:14.632203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b771947-86df-4e4d-af83-56644d623995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:14.632481 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b771947-86df-4e4d-af83-56644d623995 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:14.633144 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b771947-86df-4e4d-af83-56644d623995 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:14.635120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b771947-86df-4e4d-af83-56644d623995 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:14.650778 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:DELETE=15,keystone:SELECT=81,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:14.666971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bca5a1-ad35-4929-a018-dd8b45f1c249 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:14.669220 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-84bca5a1-ad35-4929-a018-dd8b45f1c249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:14.683473 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bca5a1-ad35-4929-a018-dd8b45f1c249 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:14.688997 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bea93b-527e-47f3-828d-5b5be724280f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:14.689745 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bea93b-527e-47f3-828d-5b5be724280f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:14.690129 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bea93b-527e-47f3-828d-5b5be724280f admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:14.690466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bea93b-527e-47f3-828d-5b5be724280f admin admin] PATH_INFO: `/v3/projects/58d7b7da252048efae4dff8a35d92dd7` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:14.691778 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bea93b-527e-47f3-828d-5b5be724280f admin admin] RBAC: Authorizing `identity:delete_project(project_id=58d7b7da252048efae4dff8a35d92dd7)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:14.694250 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bea93b-527e-47f3-828d-5b5be724280f admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:14.729501 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1318/2617] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:19:14 2022] POST /identity/v3/projects => generated 387 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:19:14.733989 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b771947-86df-4e4d-af83-56644d623995 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:14.799246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b771947-86df-4e4d-af83-56644d623995 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:14.804177 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4dc467-b3c9-496b-9e71-9f63ef86b18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:14.804801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4dc467-b3c9-496b-9e71-9f63ef86b18c admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:14.805085 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4dc467-b3c9-496b-9e71-9f63ef86b18c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:14.805391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4dc467-b3c9-496b-9e71-9f63ef86b18c admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:14.806051 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4dc467-b3c9-496b-9e71-9f63ef86b18c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:14.808385 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4dc467-b3c9-496b-9e71-9f63ef86b18c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:14.823682 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e4bea93b-527e-47f3-828d-5b5be724280f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58d7b7da252048efae4dff8a35d92dd7', 'request_id': 'req-e4bea93b-527e-47f3-828d-5b5be724280f'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:14.830310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e4bea93b-527e-47f3-828d-5b5be724280f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58d7b7da252048efae4dff8a35d92dd7', 'request_id': 'req-e4bea93b-527e-47f3-828d-5b5be724280f'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:14.836084 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e4bea93b-527e-47f3-828d-5b5be724280f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58d7b7da252048efae4dff8a35d92dd7', 'request_id': 'req-e4bea93b-527e-47f3-828d-5b5be724280f'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:14.906409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e4bea93b-527e-47f3-828d-5b5be724280f admin admin] The token cache is being invalidate because project 58d7b7da252048efae4dff8a35d92dd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:19:14.907148 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e4bea93b-527e-47f3-828d-5b5be724280f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:14.907251 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1319/2618] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:19:14 2022] POST /identity/v3/users => generated 300 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:19:14.907433 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e4bea93b-527e-47f3-828d-5b5be724280f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:14.909160 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1300/2619] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:19:14 2022] DELETE /identity/v3/projects/58d7b7da252048efae4dff8a35d92dd7 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:19:14.912744 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4dc467-b3c9-496b-9e71-9f63ef86b18c admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:14.913435 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2e4dc467-b3c9-496b-9e71-9f63ef86b18c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:14.914030 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3f28b-f409-4d90-b8df-0d7c8d845f9e None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:14.914322 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3f28b-f409-4d90-b8df-0d7c8d845f9e None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:14.914608 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3f28b-f409-4d90-b8df-0d7c8d845f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:14.928747 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4dc467-b3c9-496b-9e71-9f63ef86b18c admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:14.938581 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-7db3f28b-f409-4d90-b8df-0d7c8d845f9e None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:14.940080 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-7db3f28b-f409-4d90-b8df-0d7c8d845f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:14.982939 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:14.987524 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b9ab1-a789-44b8-a135-03f161891e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:14.988153 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b9ab1-a789-44b8-a135-03f161891e5e admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:14.988447 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b9ab1-a789-44b8-a135-03f161891e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:14.988725 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b9ab1-a789-44b8-a135-03f161891e5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:14.989420 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b9ab1-a789-44b8-a135-03f161891e5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:14.991074 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282b9ab1-a789-44b8-a135-03f161891e5e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:15.001004 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1320/2620] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:19:15.001540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1301/2621] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:19:15.005156 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b9ab1-a789-44b8-a135-03f161891e5e admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:15.008801 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fbeee-93b2-4fd5-ae4a-ff115a24fd90 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:15.009088 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fbeee-93b2-4fd5-ae4a-ff115a24fd90 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:15.009374 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894fbeee-93b2-4fd5-ae4a-ff115a24fd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:15.015140 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b9ab1-a789-44b8-a135-03f161891e5e admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:15.019935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330f3d6-9972-45ec-ae1a-b3d27bea9764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:15.020594 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330f3d6-9972-45ec-ae1a-b3d27bea9764 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:15.020874 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330f3d6-9972-45ec-ae1a-b3d27bea9764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:15.021149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8330f3d6-9972-45ec-ae1a-b3d27bea9764 admin admin] PATH_INFO: `/v3/projects/da7c8eb373494310aba867220ecbd276/users/0cb0f50ab2f0449595e4626b7dc6b490/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:15.029612 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-894fbeee-93b2-4fd5-ae4a-ff115a24fd90 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:15.030857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-894fbeee-93b2-4fd5-ae4a-ff115a24fd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:15.037762 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8330f3d6-9972-45ec-ae1a-b3d27bea9764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da7c8eb373494310aba867220ecbd276, user_id=0cb0f50ab2f0449595e4626b7dc6b490, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:15.038898 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1302/2622] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:19:15.040527 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8330f3d6-9972-45ec-ae1a-b3d27bea9764 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:15.043911 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fbeee-93b2-4fd5-ae4a-ff115a24fd90 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:15.053764 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894fbeee-93b2-4fd5-ae4a-ff115a24fd90 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:15.059088 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468576e1-48a7-467a-bd7a-3b3131ae3df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:15.059697 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468576e1-48a7-467a-bd7a-3b3131ae3df9 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:15.059981 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468576e1-48a7-467a-bd7a-3b3131ae3df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:15.060260 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468576e1-48a7-467a-bd7a-3b3131ae3df9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:15.060955 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468576e1-48a7-467a-bd7a-3b3131ae3df9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:19:15.061278 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468576e1-48a7-467a-bd7a-3b3131ae3df9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:15.063020 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468576e1-48a7-467a-bd7a-3b3131ae3df9 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:15.074862 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1303/2623] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:19:15.079425 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468576e1-48a7-467a-bd7a-3b3131ae3df9 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:15.089275 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468576e1-48a7-467a-bd7a-3b3131ae3df9 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:15.094737 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39947cce-a27a-49d9-9b71-fe5f8b3d77b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:15.095337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39947cce-a27a-49d9-9b71-fe5f8b3d77b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:15.095618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39947cce-a27a-49d9-9b71-fe5f8b3d77b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:15.095895 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39947cce-a27a-49d9-9b71-fe5f8b3d77b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:15.096554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39947cce-a27a-49d9-9b71-fe5f8b3d77b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:15.098484 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39947cce-a27a-49d9-9b71-fe5f8b3d77b1 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:15.107610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1321/2624] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:15 2022] PUT /identity/v3/projects/da7c8eb373494310aba867220ecbd276/users/0cb0f50ab2f0449595e4626b7dc6b490/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:19:15.113152 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330f3d6-9972-45ec-ae1a-b3d27bea9764 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:15.172254 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:15.176000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8330f3d6-9972-45ec-ae1a-b3d27bea9764 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:15.180736 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d93d5b-9b08-4dc9-8ef8-1da1dc8d1394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:15.181368 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d93d5b-9b08-4dc9-8ef8-1da1dc8d1394 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:15.181936 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d93d5b-9b08-4dc9-8ef8-1da1dc8d1394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:15.182220 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d93d5b-9b08-4dc9-8ef8-1da1dc8d1394 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:15.182545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1304/2625] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:19:15 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:19:15.183096 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d93d5b-9b08-4dc9-8ef8-1da1dc8d1394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:15.184740 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d93d5b-9b08-4dc9-8ef8-1da1dc8d1394 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:15.186803 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39947cce-a27a-49d9-9b71-fe5f8b3d77b1 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:15.195628 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1322/2626] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:19:15.205996 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d93d5b-9b08-4dc9-8ef8-1da1dc8d1394 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:15.216566 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d93d5b-9b08-4dc9-8ef8-1da1dc8d1394 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:15.252832 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39947cce-a27a-49d9-9b71-fe5f8b3d77b1 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:15.257664 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066de895-50fa-48fc-837c-ac2ec409f1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:15.258330 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066de895-50fa-48fc-837c-ac2ec409f1a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:15.258616 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066de895-50fa-48fc-837c-ac2ec409f1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:15.258895 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066de895-50fa-48fc-837c-ac2ec409f1a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:15.259602 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066de895-50fa-48fc-837c-ac2ec409f1a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:15.261524 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066de895-50fa-48fc-837c-ac2ec409f1a0 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:15.273111 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540e332-4875-4acc-9749-7f095a43906a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:15.273787 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540e332-4875-4acc-9749-7f095a43906a admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:15.274104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540e332-4875-4acc-9749-7f095a43906a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:15.274382 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540e332-4875-4acc-9749-7f095a43906a admin admin] PATH_INFO: `/v3/projects/da7c8eb373494310aba867220ecbd276/users/0cb0f50ab2f0449595e4626b7dc6b490/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:15.276443 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540e332-4875-4acc-9749-7f095a43906a admin admin] RBAC: Authorizing `identity:create_grant(project_id=da7c8eb373494310aba867220ecbd276, user_id=0cb0f50ab2f0449595e4626b7dc6b490, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:15.280912 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540e332-4875-4acc-9749-7f095a43906a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:15.292151 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1323/2627] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:15 2022] PUT /identity/v3/projects/da7c8eb373494310aba867220ecbd276/users/0cb0f50ab2f0449595e4626b7dc6b490/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:19:15.303558 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76865df-cb14-4c17-b601-7693722bacf0 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:15.303848 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76865df-cb14-4c17-b601-7693722bacf0 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:15.304131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76865df-cb14-4c17-b601-7693722bacf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:15.332437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f76865df-cb14-4c17-b601-7693722bacf0 None None] MFA Rules not processed for user `0cb0f50ab2f0449595e4626b7dc6b490`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:15.333674 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f76865df-cb14-4c17-b601-7693722bacf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:15.347386 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1305/2628] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:19:15 2022] POST /identity/v3/users => generated 292 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:19:15.351570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066de895-50fa-48fc-837c-ac2ec409f1a0 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:15.361719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066de895-50fa-48fc-837c-ac2ec409f1a0 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:15.404280 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1324/2629] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:15 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:19:15.417816 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76865df-cb14-4c17-b601-7693722bacf0 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:15.419351 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f76865df-cb14-4c17-b601-7693722bacf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:15.421227 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f273226-e71d-4807-b38d-fa231491dd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:15.421860 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f273226-e71d-4807-b38d-fa231491dd7a admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:15.422181 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f273226-e71d-4807-b38d-fa231491dd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:15.422456 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f273226-e71d-4807-b38d-fa231491dd7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:15.423374 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f273226-e71d-4807-b38d-fa231491dd7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:15.425231 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f273226-e71d-4807-b38d-fa231491dd7a admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:15.437217 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1306/2630] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:19:15.441570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f273226-e71d-4807-b38d-fa231491dd7a admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:15.452140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f273226-e71d-4807-b38d-fa231491dd7a admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:15.457196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b525401-8193-4492-8b11-e2b0e5af3bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:15.457792 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b525401-8193-4492-8b11-e2b0e5af3bfd admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:15.458134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b525401-8193-4492-8b11-e2b0e5af3bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:15.458420 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b525401-8193-4492-8b11-e2b0e5af3bfd admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:15.459117 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b525401-8193-4492-8b11-e2b0e5af3bfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:15.460748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b525401-8193-4492-8b11-e2b0e5af3bfd admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:15.471542 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1307/2631] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:19:15.475671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b525401-8193-4492-8b11-e2b0e5af3bfd admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:15.478457 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76865df-cb14-4c17-b601-7693722bacf0 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:15.482262 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25c953-680a-4b08-a04e-8f456246c82a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:15.482996 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25c953-680a-4b08-a04e-8f456246c82a service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:15.483275 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25c953-680a-4b08-a04e-8f456246c82a service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:15.483553 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d25c953-680a-4b08-a04e-8f456246c82a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:15.485642 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b525401-8193-4492-8b11-e2b0e5af3bfd admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:15.490350 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8137aa-898f-4992-8005-2e017166aecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:15.491002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8137aa-898f-4992-8005-2e017166aecb admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:15.491284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8137aa-898f-4992-8005-2e017166aecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:15.491560 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8137aa-898f-4992-8005-2e017166aecb admin admin] PATH_INFO: `/v3/projects/213d4ee9d8f34f13a532fde3f4ef252e/users/74fcf1c35a2d496e93e0aa305277c6fc/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:15.493573 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d25c953-680a-4b08-a04e-8f456246c82a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:15.495106 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d25c953-680a-4b08-a04e-8f456246c82a service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:15.505049 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1325/2632] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:15 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:19:15.507939 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8137aa-898f-4992-8005-2e017166aecb admin admin] RBAC: Authorizing `identity:create_grant(project_id=213d4ee9d8f34f13a532fde3f4ef252e, user_id=74fcf1c35a2d496e93e0aa305277c6fc, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:15.510634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8137aa-898f-4992-8005-2e017166aecb admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:15.514492 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:15.572559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25c953-680a-4b08-a04e-8f456246c82a service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:15.583912 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d25c953-680a-4b08-a04e-8f456246c82a service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:15.586850 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1308/2633] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:15 2022] PUT /identity/v3/projects/213d4ee9d8f34f13a532fde3f4ef252e/users/74fcf1c35a2d496e93e0aa305277c6fc/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:15.596757 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:15.600000 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead1f53-10b1-4c1d-8da3-8f98f66089fb None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:15.600290 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead1f53-10b1-4c1d-8da3-8f98f66089fb None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:15.600565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead1f53-10b1-4c1d-8da3-8f98f66089fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:15.631595 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-bead1f53-10b1-4c1d-8da3-8f98f66089fb None None] MFA Rules not processed for user `74fcf1c35a2d496e93e0aa305277c6fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:15.632887 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-bead1f53-10b1-4c1d-8da3-8f98f66089fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:15.644866 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef6e9f-5976-4c99-9dc4-8250e0e1ff60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:15.645515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef6e9f-5976-4c99-9dc4-8250e0e1ff60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:15.645801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef6e9f-5976-4c99-9dc4-8250e0e1ff60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:15.646112 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef6e9f-5976-4c99-9dc4-8250e0e1ff60 admin admin] PATH_INFO: `/v3/users/28a9891a6857409b971e526c91bb2e72` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:15.647322 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef6e9f-5976-4c99-9dc4-8250e0e1ff60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28a9891a6857409b971e526c91bb2e72)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:15.649257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef6e9f-5976-4c99-9dc4-8250e0e1ff60 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:15.700255 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1309/2634] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:19:15.704706 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead1f53-10b1-4c1d-8da3-8f98f66089fb None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:15.714869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead1f53-10b1-4c1d-8da3-8f98f66089fb None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:15.719544 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51c308-be73-4a23-8540-d69dd97fdeab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:15.720147 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51c308-be73-4a23-8540-d69dd97fdeab admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:15.720430 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51c308-be73-4a23-8540-d69dd97fdeab admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:15.720705 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a51c308-be73-4a23-8540-d69dd97fdeab admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:15.721372 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51c308-be73-4a23-8540-d69dd97fdeab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:15.723296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a51c308-be73-4a23-8540-d69dd97fdeab admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:15.833659 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1310/2635] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:19:15 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:19:15.837559 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51c308-be73-4a23-8540-d69dd97fdeab admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:15.858780 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e1ef6e9f-5976-4c99-9dc4-8250e0e1ff60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28a9891a6857409b971e526c91bb2e72'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:15.866014 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e1ef6e9f-5976-4c99-9dc4-8250e0e1ff60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28a9891a6857409b971e526c91bb2e72'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:15.866695 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e1ef6e9f-5976-4c99-9dc4-8250e0e1ff60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28a9891a6857409b971e526c91bb2e72'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:15.878820 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e1ef6e9f-5976-4c99-9dc4-8250e0e1ff60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28a9891a6857409b971e526c91bb2e72'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:15.879503 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-e1ef6e9f-5976-4c99-9dc4-8250e0e1ff60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28a9891a6857409b971e526c91bb2e72'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:15.903678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a51c308-be73-4a23-8540-d69dd97fdeab admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:15.908436 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd730972-8f56-4e80-9701-790f55cab32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:15.909033 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd730972-8f56-4e80-9701-790f55cab32a admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:15.909322 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd730972-8f56-4e80-9701-790f55cab32a admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:15.909621 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd730972-8f56-4e80-9701-790f55cab32a admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:15.910333 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd730972-8f56-4e80-9701-790f55cab32a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:15.912251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd730972-8f56-4e80-9701-790f55cab32a admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:15.941818 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1326/2636] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:19:15 2022] DELETE /identity/v3/users/28a9891a6857409b971e526c91bb2e72 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:19:15.946559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b814c73-e7f8-45b2-acf3-207006114f0f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:15.947016 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b814c73-e7f8-45b2-acf3-207006114f0f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:15.947301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b814c73-e7f8-45b2-acf3-207006114f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:15.967431 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-2b814c73-e7f8-45b2-acf3-207006114f0f None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:15.968646 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2b814c73-e7f8-45b2-acf3-207006114f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:16.029075 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1311/2637] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:19:15 2022] POST /identity/v3/users => generated 299 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:19:16.029297 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1327/2638] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:19:16.030523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd730972-8f56-4e80-9701-790f55cab32a admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:16.032278 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-cd730972-8f56-4e80-9701-790f55cab32a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:16.036249 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b814c73-e7f8-45b2-acf3-207006114f0f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:16.037715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2b814c73-e7f8-45b2-acf3-207006114f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:16.044835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd730972-8f56-4e80-9701-790f55cab32a admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:16.049633 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917aa8ca-4379-405f-85fd-e4111c3470f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:16.050269 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917aa8ca-4379-405f-85fd-e4111c3470f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:16.050555 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917aa8ca-4379-405f-85fd-e4111c3470f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:16.050832 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917aa8ca-4379-405f-85fd-e4111c3470f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:16.051566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917aa8ca-4379-405f-85fd-e4111c3470f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:16.053202 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917aa8ca-4379-405f-85fd-e4111c3470f6 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:16.063786 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1312/2639] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:19:16.068015 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917aa8ca-4379-405f-85fd-e4111c3470f6 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:16.078923 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917aa8ca-4379-405f-85fd-e4111c3470f6 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:16.082966 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:16.083820 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb1486b-13d9-4408-81d5-b7cd1ee0c629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:16.084440 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb1486b-13d9-4408-81d5-b7cd1ee0c629 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:16.084722 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb1486b-13d9-4408-81d5-b7cd1ee0c629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:16.085012 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb1486b-13d9-4408-81d5-b7cd1ee0c629 admin admin] PATH_INFO: `/v3/projects/91e25fa6f3e344f2891ae245c92e481d/users/2109a825c63e431a951b5cdb31b9ae23/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:16.101416 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b814c73-e7f8-45b2-acf3-207006114f0f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:16.101685 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb1486b-13d9-4408-81d5-b7cd1ee0c629 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91e25fa6f3e344f2891ae245c92e481d, user_id=2109a825c63e431a951b5cdb31b9ae23, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:16.104452 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb1486b-13d9-4408-81d5-b7cd1ee0c629 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:16.105180 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f8320-7ec1-4855-9942-75c46aee706a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:16.105801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f8320-7ec1-4855-9942-75c46aee706a service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:16.106115 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f8320-7ec1-4855-9942-75c46aee706a service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:16.106398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f8320-7ec1-4855-9942-75c46aee706a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:16.116301 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f8320-7ec1-4855-9942-75c46aee706a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:16.117950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208f8320-7ec1-4855-9942-75c46aee706a service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:16.127468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1328/2640] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:19:16.203767 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1313/2641] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:16 2022] PUT /identity/v3/projects/91e25fa6f3e344f2891ae245c92e481d/users/2109a825c63e431a951b5cdb31b9ae23/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:19:16.207765 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f8320-7ec1-4855-9942-75c46aee706a service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:16.217915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f8320-7ec1-4855-9942-75c46aee706a service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:16.275874 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad2508-2bbc-46bb-bd75-0d49095b3851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:16.276494 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad2508-2bbc-46bb-bd75-0d49095b3851 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:16.276781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad2508-2bbc-46bb-bd75-0d49095b3851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:16.277055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad2508-2bbc-46bb-bd75-0d49095b3851 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:16.278365 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad2508-2bbc-46bb-bd75-0d49095b3851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:16.279964 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad2508-2bbc-46bb-bd75-0d49095b3851 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:16.290381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1329/2642] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:19:16.301414 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb1486b-13d9-4408-81d5-b7cd1ee0c629 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:16.313343 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb1486b-13d9-4408-81d5-b7cd1ee0c629 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:16.318144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ff2c1-2a09-4771-91e2-38d98380b4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:16.318936 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ff2c1-2a09-4771-91e2-38d98380b4da admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:16.319227 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ff2c1-2a09-4771-91e2-38d98380b4da admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:16.319505 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62ff2c1-2a09-4771-91e2-38d98380b4da admin admin] PATH_INFO: `/v3/projects/91e25fa6f3e344f2891ae245c92e481d/users/2109a825c63e431a951b5cdb31b9ae23/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:16.321553 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ff2c1-2a09-4771-91e2-38d98380b4da admin admin] RBAC: Authorizing `identity:create_grant(project_id=91e25fa6f3e344f2891ae245c92e481d, user_id=2109a825c63e431a951b5cdb31b9ae23, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:16.324221 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62ff2c1-2a09-4771-91e2-38d98380b4da admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:16.339765 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1314/2643] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:16 2022] PUT /identity/v3/projects/91e25fa6f3e344f2891ae245c92e481d/users/2109a825c63e431a951b5cdb31b9ae23/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:16.409125 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d149bf7-8373-4d13-948e-eadc12755c14 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:16.409424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d149bf7-8373-4d13-948e-eadc12755c14 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:16.409708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d149bf7-8373-4d13-948e-eadc12755c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:16.439242 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-6d149bf7-8373-4d13-948e-eadc12755c14 None None] MFA Rules not processed for user `2109a825c63e431a951b5cdb31b9ae23`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:16.440559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6d149bf7-8373-4d13-948e-eadc12755c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:16.510960 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1330/2644] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:16.522259 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ff2c1-2a09-4771-91e2-38d98380b4da admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:16.523936 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-c62ff2c1-2a09-4771-91e2-38d98380b4da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:16.529242 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:16.548445 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d149bf7-8373-4d13-948e-eadc12755c14 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:16.551539 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6d149bf7-8373-4d13-948e-eadc12755c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:16.601261 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62ff2c1-2a09-4771-91e2-38d98380b4da admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:16.605065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf6429-50c0-47d2-a405-f87b8469ad27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:16.605743 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf6429-50c0-47d2-a405-f87b8469ad27 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:16.606060 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf6429-50c0-47d2-a405-f87b8469ad27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:16.606369 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcf6429-50c0-47d2-a405-f87b8469ad27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:16.610993 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4fcf6429-50c0-47d2-a405-f87b8469ad27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:16.626019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d149bf7-8373-4d13-948e-eadc12755c14 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:16.631734 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461738f-7505-4a30-98c4-e6c58f74d71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:16.632608 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461738f-7505-4a30-98c4-e6c58f74d71f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:16.632895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461738f-7505-4a30-98c4-e6c58f74d71f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:16.633175 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461738f-7505-4a30-98c4-e6c58f74d71f admin admin] PATH_INFO: `/v3/projects/3901f7fcbb76480e8703b4898a09500d` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:16.634414 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461738f-7505-4a30-98c4-e6c58f74d71f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3901f7fcbb76480e8703b4898a09500d)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:16.636557 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461738f-7505-4a30-98c4-e6c58f74d71f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:16.639051 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=100 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:16.669411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf6429-50c0-47d2-a405-f87b8469ad27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:16.670885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcf6429-50c0-47d2-a405-f87b8469ad27 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:16.680025 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1315/2645] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:19:16.759544 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f461738f-7505-4a30-98c4-e6c58f74d71f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3901f7fcbb76480e8703b4898a09500d', 'request_id': 'req-f461738f-7505-4a30-98c4-e6c58f74d71f'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:16.766986 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f461738f-7505-4a30-98c4-e6c58f74d71f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3901f7fcbb76480e8703b4898a09500d', 'request_id': 'req-f461738f-7505-4a30-98c4-e6c58f74d71f'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:16.855186 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f461738f-7505-4a30-98c4-e6c58f74d71f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3901f7fcbb76480e8703b4898a09500d', 'request_id': 'req-f461738f-7505-4a30-98c4-e6c58f74d71f'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:16.866932 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f461738f-7505-4a30-98c4-e6c58f74d71f admin admin] The token cache is being invalidate because project 3901f7fcbb76480e8703b4898a09500d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:19:16.867659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f461738f-7505-4a30-98c4-e6c58f74d71f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:16.867940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-f461738f-7505-4a30-98c4-e6c58f74d71f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:16.869615 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1331/2646] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:19:16 2022] DELETE /identity/v3/projects/3901f7fcbb76480e8703b4898a09500d => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:19:16.874048 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf6429-50c0-47d2-a405-f87b8469ad27 service cinder] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:16.889260 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcf6429-50c0-47d2-a405-f87b8469ad27 service cinder] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:16.949203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a2a9c-6378-4f45-9c5f-a46c547f801e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:16.950446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a2a9c-6378-4f45-9c5f-a46c547f801e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:16.950727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a2a9c-6378-4f45-9c5f-a46c547f801e admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:16.950999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a2a9c-6378-4f45-9c5f-a46c547f801e admin admin] PATH_INFO: `/v3/users/0cb0f50ab2f0449595e4626b7dc6b490` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:16.952351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283a2a9c-6378-4f45-9c5f-a46c547f801e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cb0f50ab2f0449595e4626b7dc6b490)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:16.954395 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283a2a9c-6378-4f45-9c5f-a46c547f801e admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:17.156546 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-283a2a9c-6378-4f45-9c5f-a46c547f801e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cb0f50ab2f0449595e4626b7dc6b490'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:17.164439 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-283a2a9c-6378-4f45-9c5f-a46c547f801e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cb0f50ab2f0449595e4626b7dc6b490'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:17.165158 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-283a2a9c-6378-4f45-9c5f-a46c547f801e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cb0f50ab2f0449595e4626b7dc6b490'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:17.168259 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:17.237373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-283a2a9c-6378-4f45-9c5f-a46c547f801e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cb0f50ab2f0449595e4626b7dc6b490'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:17.250668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-283a2a9c-6378-4f45-9c5f-a46c547f801e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cb0f50ab2f0449595e4626b7dc6b490'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:17.256606 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1316/2647] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:19:16 2022] DELETE /identity/v3/users/0cb0f50ab2f0449595e4626b7dc6b490 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:19:17.713655 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461738f-7505-4a30-98c4-e6c58f74d71f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:17.715123 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f461738f-7505-4a30-98c4-e6c58f74d71f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:17.727918 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461738f-7505-4a30-98c4-e6c58f74d71f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:17.786411 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6576ed8-b61b-4279-8fdf-83fe3aa853d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:17.787061 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6576ed8-b61b-4279-8fdf-83fe3aa853d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:17.787337 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6576ed8-b61b-4279-8fdf-83fe3aa853d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:17.787609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6576ed8-b61b-4279-8fdf-83fe3aa853d5 admin admin] PATH_INFO: `/v3/projects/da7c8eb373494310aba867220ecbd276` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:17.788754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6576ed8-b61b-4279-8fdf-83fe3aa853d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da7c8eb373494310aba867220ecbd276)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:17.790676 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6576ed8-b61b-4279-8fdf-83fe3aa853d5 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:17.917644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c6576ed8-b61b-4279-8fdf-83fe3aa853d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da7c8eb373494310aba867220ecbd276', 'request_id': 'req-c6576ed8-b61b-4279-8fdf-83fe3aa853d5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:17.924641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c6576ed8-b61b-4279-8fdf-83fe3aa853d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da7c8eb373494310aba867220ecbd276', 'request_id': 'req-c6576ed8-b61b-4279-8fdf-83fe3aa853d5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:18.123850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c6576ed8-b61b-4279-8fdf-83fe3aa853d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da7c8eb373494310aba867220ecbd276', 'request_id': 'req-c6576ed8-b61b-4279-8fdf-83fe3aa853d5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:18.136559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c6576ed8-b61b-4279-8fdf-83fe3aa853d5 admin admin] The token cache is being invalidate because project da7c8eb373494310aba867220ecbd276 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:19:18.137924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c6576ed8-b61b-4279-8fdf-83fe3aa853d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:18.139050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-c6576ed8-b61b-4279-8fdf-83fe3aa853d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:18.141523 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1332/2648] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:19:17 2022] DELETE /identity/v3/projects/da7c8eb373494310aba867220ecbd276 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:18.148280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20679d-0c74-4d60-97c6-09ae95c875e3 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:18.148855 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20679d-0c74-4d60-97c6-09ae95c875e3 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:18.149287 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20679d-0c74-4d60-97c6-09ae95c875e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:18.178272 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-dd20679d-0c74-4d60-97c6-09ae95c875e3 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:18.178272 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-dd20679d-0c74-4d60-97c6-09ae95c875e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:18.238997 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1317/2649] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:18.247098 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95101a9a-ee7f-4c70-9894-9d775add01d1 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:18.247334 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95101a9a-ee7f-4c70-9894-9d775add01d1 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:18.247608 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95101a9a-ee7f-4c70-9894-9d775add01d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:18.267852 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-95101a9a-ee7f-4c70-9894-9d775add01d1 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:18.269126 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-95101a9a-ee7f-4c70-9894-9d775add01d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:18.277285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1333/2650] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:18.282084 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20679d-0c74-4d60-97c6-09ae95c875e3 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:18.292113 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20679d-0c74-4d60-97c6-09ae95c875e3 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:18.297763 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be49a4-6b6b-4d1c-8797-71fc67a45167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:18.298409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be49a4-6b6b-4d1c-8797-71fc67a45167 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:18.298693 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be49a4-6b6b-4d1c-8797-71fc67a45167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:18.298999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18be49a4-6b6b-4d1c-8797-71fc67a45167 admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:18.299736 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18be49a4-6b6b-4d1c-8797-71fc67a45167 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:19:18.300044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18be49a4-6b6b-4d1c-8797-71fc67a45167 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:18.301648 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18be49a4-6b6b-4d1c-8797-71fc67a45167 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:18.314704 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1318/2651] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:19:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:19:18.319031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95101a9a-ee7f-4c70-9894-9d775add01d1 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:18.329389 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95101a9a-ee7f-4c70-9894-9d775add01d1 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:18.334180 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465a0ab-e674-443e-9934-841b7f5d0867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:18.337786 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465a0ab-e674-443e-9934-841b7f5d0867 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:18.337786 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465a0ab-e674-443e-9934-841b7f5d0867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:18.337786 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465a0ab-e674-443e-9934-841b7f5d0867 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:18.337786 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465a0ab-e674-443e-9934-841b7f5d0867 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:18.338176 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465a0ab-e674-443e-9934-841b7f5d0867 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:18.351579 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:18.445576 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1334/2652] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:19:18 2022] POST /identity/v3/projects => generated 399 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:19:18.453846 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be49a4-6b6b-4d1c-8797-71fc67a45167 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:18.461140 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18be49a4-6b6b-4d1c-8797-71fc67a45167 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:18.523701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc56f7-753b-4e7b-ab88-b7afeccc8a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:18.524118 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc56f7-753b-4e7b-ab88-b7afeccc8a64 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:18.524399 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc56f7-753b-4e7b-ab88-b7afeccc8a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:18.524677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc56f7-753b-4e7b-ab88-b7afeccc8a64 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:18.525371 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc56f7-753b-4e7b-ab88-b7afeccc8a64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:18.527910 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc56f7-753b-4e7b-ab88-b7afeccc8a64 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:18.620145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1319/2653] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:19:18 2022] POST /identity/v3/users => generated 300 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:19:18.624750 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465a0ab-e674-443e-9934-841b7f5d0867 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:18.635241 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465a0ab-e674-443e-9934-841b7f5d0867 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:18.640549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de489a9-8465-4a2e-9aa0-460c5e9e95a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:18.641163 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de489a9-8465-4a2e-9aa0-460c5e9e95a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:18.641444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de489a9-8465-4a2e-9aa0-460c5e9e95a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:18.641746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de489a9-8465-4a2e-9aa0-460c5e9e95a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:18.642469 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de489a9-8465-4a2e-9aa0-460c5e9e95a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:18.644079 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de489a9-8465-4a2e-9aa0-460c5e9e95a8 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:18.654343 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1335/2654] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:19:18.658557 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc56f7-753b-4e7b-ab88-b7afeccc8a64 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:18.668618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc56f7-753b-4e7b-ab88-b7afeccc8a64 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:18.673863 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd1d91-f13e-4ef4-8337-a58b5247d55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:18.677159 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd1d91-f13e-4ef4-8337-a58b5247d55a admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:18.677159 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd1d91-f13e-4ef4-8337-a58b5247d55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:18.677159 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd1d91-f13e-4ef4-8337-a58b5247d55a admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:18.677159 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd1d91-f13e-4ef4-8337-a58b5247d55a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:18.677579 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd1d91-f13e-4ef4-8337-a58b5247d55a admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:18.689858 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1320/2655] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:19:18.694215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de489a9-8465-4a2e-9aa0-460c5e9e95a8 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:18.705302 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de489a9-8465-4a2e-9aa0-460c5e9e95a8 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:18.709685 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a017c-00e5-47ff-bfc7-5d27a36a7384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:18.710383 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a017c-00e5-47ff-bfc7-5d27a36a7384 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:18.710666 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a017c-00e5-47ff-bfc7-5d27a36a7384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:18.710948 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222a017c-00e5-47ff-bfc7-5d27a36a7384 admin admin] PATH_INFO: `/v3/projects/c17c760f9f4a4fe6a547b4c473f36165/users/c64ce3cc75284a31a725f7e8f259da18/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:18.725498 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a017c-00e5-47ff-bfc7-5d27a36a7384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c17c760f9f4a4fe6a547b4c473f36165, user_id=c64ce3cc75284a31a725f7e8f259da18, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:18.728196 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222a017c-00e5-47ff-bfc7-5d27a36a7384 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:18.802192 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1336/2656] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:18 2022] PUT /identity/v3/projects/c17c760f9f4a4fe6a547b4c473f36165/users/c64ce3cc75284a31a725f7e8f259da18/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:18.816284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df1259-72a7-4533-a59d-3d8b0a0af6fa None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:18.816593 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df1259-72a7-4533-a59d-3d8b0a0af6fa None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:18.816843 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df1259-72a7-4533-a59d-3d8b0a0af6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:18.847465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-c4df1259-72a7-4533-a59d-3d8b0a0af6fa None None] MFA Rules not processed for user `c64ce3cc75284a31a725f7e8f259da18`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:18.848687 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-c4df1259-72a7-4533-a59d-3d8b0a0af6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:18.878920 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:18.916525 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1321/2657] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:18 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:18.931795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a017c-00e5-47ff-bfc7-5d27a36a7384 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:19.007272 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222a017c-00e5-47ff-bfc7-5d27a36a7384 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:19.013914 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f2229-5fe2-49d6-8186-4f9451a129ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:19.014626 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f2229-5fe2-49d6-8186-4f9451a129ac admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:19.014905 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f2229-5fe2-49d6-8186-4f9451a129ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:19.015177 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f2229-5fe2-49d6-8186-4f9451a129ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:19.015848 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9f2229-5fe2-49d6-8186-4f9451a129ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:19.017750 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9f2229-5fe2-49d6-8186-4f9451a129ac admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:19.119222 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1337/2658] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:19:18 2022] POST /identity/v3/projects => generated 401 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:19:19.123235 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df1259-72a7-4533-a59d-3d8b0a0af6fa None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:19.134433 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df1259-72a7-4533-a59d-3d8b0a0af6fa None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:19.196251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4148db3-ad5d-4a13-80bb-f06d5a76adbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:19.196929 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4148db3-ad5d-4a13-80bb-f06d5a76adbb admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:19.197044 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4148db3-ad5d-4a13-80bb-f06d5a76adbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:19.197318 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4148db3-ad5d-4a13-80bb-f06d5a76adbb admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:19.198040 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4148db3-ad5d-4a13-80bb-f06d5a76adbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:19.200506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4148db3-ad5d-4a13-80bb-f06d5a76adbb admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:19.272850 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1322/2659] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:19:19 2022] POST /identity/v3/users => generated 307 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:19:19.276853 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f2229-5fe2-49d6-8186-4f9451a129ac admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:19.287993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9f2229-5fe2-49d6-8186-4f9451a129ac admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:19.292742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a72eec-5369-41a7-ab4e-e9ad2d34d611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:19.293427 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a72eec-5369-41a7-ab4e-e9ad2d34d611 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:19.293710 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a72eec-5369-41a7-ab4e-e9ad2d34d611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:19.294676 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a72eec-5369-41a7-ab4e-e9ad2d34d611 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:19.295133 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a72eec-5369-41a7-ab4e-e9ad2d34d611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:19.297140 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a72eec-5369-41a7-ab4e-e9ad2d34d611 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:19.309474 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1338/2660] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:19:19.313662 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4148db3-ad5d-4a13-80bb-f06d5a76adbb admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:19.323666 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4148db3-ad5d-4a13-80bb-f06d5a76adbb admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:19.329034 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516c2f5-9f78-4dbf-897a-e7f23b7c193d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:19.329702 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516c2f5-9f78-4dbf-897a-e7f23b7c193d admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:19.330020 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516c2f5-9f78-4dbf-897a-e7f23b7c193d admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:19.330329 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516c2f5-9f78-4dbf-897a-e7f23b7c193d admin admin] PATH_INFO: `/v3/projects/f0f3e77a92e5444690f0f4bc096d0841/users/87f739b542d440daa234485668dabca2/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:19.346278 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516c2f5-9f78-4dbf-897a-e7f23b7c193d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0f3e77a92e5444690f0f4bc096d0841, user_id=87f739b542d440daa234485668dabca2, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:19.349093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9516c2f5-9f78-4dbf-897a-e7f23b7c193d admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:19.421307 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1323/2661] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:19 2022] PUT /identity/v3/projects/f0f3e77a92e5444690f0f4bc096d0841/users/87f739b542d440daa234485668dabca2/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:19:19.427485 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a72eec-5369-41a7-ab4e-e9ad2d34d611 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:19.448254 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a72eec-5369-41a7-ab4e-e9ad2d34d611 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:19.577369 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760e0f5-1302-4c27-87ef-a24b6dd9ecd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:19.580315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760e0f5-1302-4c27-87ef-a24b6dd9ecd9 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:19.580315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760e0f5-1302-4c27-87ef-a24b6dd9ecd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:19.580315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760e0f5-1302-4c27-87ef-a24b6dd9ecd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:19.580880 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760e0f5-1302-4c27-87ef-a24b6dd9ecd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:19.584060 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760e0f5-1302-4c27-87ef-a24b6dd9ecd9 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:19.609829 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516c2f5-9f78-4dbf-897a-e7f23b7c193d admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:19.622576 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516c2f5-9f78-4dbf-897a-e7f23b7c193d admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:19.625004 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1339/2662] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:19 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:19:19.627838 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251850c-dd9c-45ee-b487-963e113265f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:19.628929 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251850c-dd9c-45ee-b487-963e113265f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:19.629303 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251850c-dd9c-45ee-b487-963e113265f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:19.630305 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251850c-dd9c-45ee-b487-963e113265f3 admin admin] PATH_INFO: `/v3/projects/f0f3e77a92e5444690f0f4bc096d0841/users/87f739b542d440daa234485668dabca2/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:19.631951 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8251850c-dd9c-45ee-b487-963e113265f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0f3e77a92e5444690f0f4bc096d0841, user_id=87f739b542d440daa234485668dabca2, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:19.634888 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8251850c-dd9c-45ee-b487-963e113265f3 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:19.647604 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1324/2663] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:19 2022] PUT /identity/v3/projects/f0f3e77a92e5444690f0f4bc096d0841/users/87f739b542d440daa234485668dabca2/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:19:19.719366 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793df28-a1b8-455c-9e0f-93aae509c678 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:19.719638 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793df28-a1b8-455c-9e0f-93aae509c678 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:19.719908 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793df28-a1b8-455c-9e0f-93aae509c678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:19.749271 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-6793df28-a1b8-455c-9e0f-93aae509c678 None None] MFA Rules not processed for user `87f739b542d440daa234485668dabca2`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:19.750534 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6793df28-a1b8-455c-9e0f-93aae509c678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:19.764490 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:19.820968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1340/2664] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:19 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:19.832029 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251850c-dd9c-45ee-b487-963e113265f3 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:19.833715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-8251850c-dd9c-45ee-b487-963e113265f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:19.902408 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251850c-dd9c-45ee-b487-963e113265f3 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:19.906229 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ead25-a207-4ae0-b84a-4874d3b50af8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:19.906854 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ead25-a207-4ae0-b84a-4874d3b50af8 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:19.907136 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ead25-a207-4ae0-b84a-4874d3b50af8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:19.907415 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ead25-a207-4ae0-b84a-4874d3b50af8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:19.917780 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ead25-a207-4ae0-b84a-4874d3b50af8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:19.919805 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ead25-a207-4ae0-b84a-4874d3b50af8 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:19.932617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1325/2665] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:19 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:19:20.185825 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793df28-a1b8-455c-9e0f-93aae509c678 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:20.197114 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793df28-a1b8-455c-9e0f-93aae509c678 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:20.203284 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ec788-8c17-4fc6-b403-8803c462118a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:20.203931 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ec788-8c17-4fc6-b403-8803c462118a service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:20.204302 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ec788-8c17-4fc6-b403-8803c462118a service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:20.204637 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ec788-8c17-4fc6-b403-8803c462118a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:20.264451 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ec788-8c17-4fc6-b403-8803c462118a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:20.265962 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ec788-8c17-4fc6-b403-8803c462118a service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:20.275437 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1341/2666] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:20 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:19:22.967369 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ead25-a207-4ae0-b84a-4874d3b50af8 service cinder] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:22.976279 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ead25-a207-4ae0-b84a-4874d3b50af8 service cinder] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:22.979900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1098c7b-bb6d-41fd-8a13-5d58a6732a61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:22.981020 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1098c7b-bb6d-41fd-8a13-5d58a6732a61 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:22.981298 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1098c7b-bb6d-41fd-8a13-5d58a6732a61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:22.981565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1098c7b-bb6d-41fd-8a13-5d58a6732a61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:22.983406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-d1098c7b-bb6d-41fd-8a13-5d58a6732a61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:23.009950 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:23.048027 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1098c7b-bb6d-41fd-8a13-5d58a6732a61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:23.049472 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1098c7b-bb6d-41fd-8a13-5d58a6732a61 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:23.059437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1326/2667] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:22 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:19:25.148663 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ec788-8c17-4fc6-b403-8803c462118a service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:25.213985 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ec788-8c17-4fc6-b403-8803c462118a service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:25.218519 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b2077-b9fe-40df-b42b-4285274a59dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:25.219161 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b2077-b9fe-40df-b42b-4285274a59dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:25.219442 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b2077-b9fe-40df-b42b-4285274a59dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:25.219720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b2077-b9fe-40df-b42b-4285274a59dd admin admin] PATH_INFO: `/v3/users/c64ce3cc75284a31a725f7e8f259da18` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:25.220873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b2077-b9fe-40df-b42b-4285274a59dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c64ce3cc75284a31a725f7e8f259da18)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:25.222815 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b2077-b9fe-40df-b42b-4285274a59dd admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:25.383425 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-470b2077-b9fe-40df-b42b-4285274a59dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c64ce3cc75284a31a725f7e8f259da18'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:25.387921 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:25.390925 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-470b2077-b9fe-40df-b42b-4285274a59dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c64ce3cc75284a31a725f7e8f259da18'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:25.391613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-470b2077-b9fe-40df-b42b-4285274a59dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c64ce3cc75284a31a725f7e8f259da18'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:25.403745 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-470b2077-b9fe-40df-b42b-4285274a59dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c64ce3cc75284a31a725f7e8f259da18'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:25.404416 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-470b2077-b9fe-40df-b42b-4285274a59dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c64ce3cc75284a31a725f7e8f259da18'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:25.494610 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1342/2668] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:19:25 2022] DELETE /identity/v3/users/c64ce3cc75284a31a725f7e8f259da18 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:25.499889 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b79e79-7f98-40ca-8013-6633c4e0a306 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:25.500406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b79e79-7f98-40ca-8013-6633c4e0a306 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:25.500525 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b79e79-7f98-40ca-8013-6633c4e0a306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:25.519804 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-d3b79e79-7f98-40ca-8013-6633c4e0a306 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:25.521018 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-d3b79e79-7f98-40ca-8013-6633c4e0a306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:25.580501 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1327/2669] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:25.590312 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b2077-b9fe-40df-b42b-4285274a59dd admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:25.591743 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-470b2077-b9fe-40df-b42b-4285274a59dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:25.650374 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b2077-b9fe-40df-b42b-4285274a59dd admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:25.654141 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69885f-7adc-426e-a7c9-aa40c4f902d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:25.654781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69885f-7adc-426e-a7c9-aa40c4f902d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:25.655065 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69885f-7adc-426e-a7c9-aa40c4f902d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:25.655346 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c69885f-7adc-426e-a7c9-aa40c4f902d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:25.664730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69885f-7adc-426e-a7c9-aa40c4f902d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:25.666194 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c69885f-7adc-426e-a7c9-aa40c4f902d7 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:25.676305 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1343/2670] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:19:26.149425 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b79e79-7f98-40ca-8013-6633c4e0a306 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:26.151126 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-d3b79e79-7f98-40ca-8013-6633c4e0a306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:26.166975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b79e79-7f98-40ca-8013-6633c4e0a306 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:26.185697 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1a012-7041-45f0-9d08-24bb540e70b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:26.186382 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1a012-7041-45f0-9d08-24bb540e70b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:26.189413 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1a012-7041-45f0-9d08-24bb540e70b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:26.189413 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea1a012-7041-45f0-9d08-24bb540e70b7 admin admin] PATH_INFO: `/v3/projects/c17c760f9f4a4fe6a547b4c473f36165` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:26.189413 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1a012-7041-45f0-9d08-24bb540e70b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c17c760f9f4a4fe6a547b4c473f36165)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:26.190088 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea1a012-7041-45f0-9d08-24bb540e70b7 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:26.317302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-6ea1a012-7041-45f0-9d08-24bb540e70b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c17c760f9f4a4fe6a547b4c473f36165', 'request_id': 'req-6ea1a012-7041-45f0-9d08-24bb540e70b7'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:26.326110 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-6ea1a012-7041-45f0-9d08-24bb540e70b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c17c760f9f4a4fe6a547b4c473f36165', 'request_id': 'req-6ea1a012-7041-45f0-9d08-24bb540e70b7'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:26.329797 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-6ea1a012-7041-45f0-9d08-24bb540e70b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c17c760f9f4a4fe6a547b4c473f36165', 'request_id': 'req-6ea1a012-7041-45f0-9d08-24bb540e70b7'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:26.443214 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-6ea1a012-7041-45f0-9d08-24bb540e70b7 admin admin] The token cache is being invalidate because project c17c760f9f4a4fe6a547b4c473f36165 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:19:26.443546 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-6ea1a012-7041-45f0-9d08-24bb540e70b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:26.444122 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-6ea1a012-7041-45f0-9d08-24bb540e70b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:26.445642 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1328/2671] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:19:26 2022] DELETE /identity/v3/projects/c17c760f9f4a4fe6a547b4c473f36165 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:26.450324 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69885f-7adc-426e-a7c9-aa40c4f902d7 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:26.460517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c69885f-7adc-426e-a7c9-aa40c4f902d7 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:26.524779 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a483ae3-490b-4bcf-8c63-f3cdde840e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:26.525429 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a483ae3-490b-4bcf-8c63-f3cdde840e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:26.525713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a483ae3-490b-4bcf-8c63-f3cdde840e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:26.526137 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a483ae3-490b-4bcf-8c63-f3cdde840e33 admin admin] PATH_INFO: `/v3/users/87f739b542d440daa234485668dabca2` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:26.527971 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a483ae3-490b-4bcf-8c63-f3cdde840e33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87f739b542d440daa234485668dabca2)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:26.529925 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a483ae3-490b-4bcf-8c63-f3cdde840e33 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:26.692446 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1a483ae3-490b-4bcf-8c63-f3cdde840e33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87f739b542d440daa234485668dabca2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:26.700862 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1a483ae3-490b-4bcf-8c63-f3cdde840e33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87f739b542d440daa234485668dabca2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:26.701599 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1a483ae3-490b-4bcf-8c63-f3cdde840e33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87f739b542d440daa234485668dabca2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:26.714157 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1a483ae3-490b-4bcf-8c63-f3cdde840e33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87f739b542d440daa234485668dabca2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:26.715493 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1a483ae3-490b-4bcf-8c63-f3cdde840e33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87f739b542d440daa234485668dabca2'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:26.717178 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:26.829708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1344/2672] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:19:26 2022] DELETE /identity/v3/users/87f739b542d440daa234485668dabca2 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:27.330975 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1a012-7041-45f0-9d08-24bb540e70b7 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:27.332517 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-6ea1a012-7041-45f0-9d08-24bb540e70b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:27.346965 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea1a012-7041-45f0-9d08-24bb540e70b7 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:27.397513 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:27.413175 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b94f8b-6819-4a96-997f-3fb7905fecd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:27.413820 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b94f8b-6819-4a96-997f-3fb7905fecd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:27.414134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b94f8b-6819-4a96-997f-3fb7905fecd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:27.414415 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b94f8b-6819-4a96-997f-3fb7905fecd9 admin admin] PATH_INFO: `/v3/projects/f0f3e77a92e5444690f0f4bc096d0841` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:27.415574 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b94f8b-6819-4a96-997f-3fb7905fecd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0f3e77a92e5444690f0f4bc096d0841)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:27.417567 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b94f8b-6819-4a96-997f-3fb7905fecd9 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:27.611496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-35b94f8b-6819-4a96-997f-3fb7905fecd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0f3e77a92e5444690f0f4bc096d0841', 'request_id': 'req-35b94f8b-6819-4a96-997f-3fb7905fecd9'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:27.617683 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-35b94f8b-6819-4a96-997f-3fb7905fecd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0f3e77a92e5444690f0f4bc096d0841', 'request_id': 'req-35b94f8b-6819-4a96-997f-3fb7905fecd9'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:27.624582 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-35b94f8b-6819-4a96-997f-3fb7905fecd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0f3e77a92e5444690f0f4bc096d0841', 'request_id': 'req-35b94f8b-6819-4a96-997f-3fb7905fecd9'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:27.701701 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-35b94f8b-6819-4a96-997f-3fb7905fecd9 admin admin] The token cache is being invalidate because project f0f3e77a92e5444690f0f4bc096d0841 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:19:27.702028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-35b94f8b-6819-4a96-997f-3fb7905fecd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:27.703096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-35b94f8b-6819-4a96-997f-3fb7905fecd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:27.704622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1329/2673] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:19:27 2022] DELETE /identity/v3/projects/f0f3e77a92e5444690f0f4bc096d0841 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:27.710848 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71871490-5c4d-4ae2-9189-d6eaba01273e None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:27.711335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71871490-5c4d-4ae2-9189-d6eaba01273e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:27.711542 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71871490-5c4d-4ae2-9189-d6eaba01273e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:27.731980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-71871490-5c4d-4ae2-9189-d6eaba01273e None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:27.733243 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-71871490-5c4d-4ae2-9189-d6eaba01273e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:27.795445 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1345/2674] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:27.803477 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b07894-e94a-48de-ab0a-8a6c7def4792 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:27.803780 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b07894-e94a-48de-ab0a-8a6c7def4792 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:27.804062 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b07894-e94a-48de-ab0a-8a6c7def4792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:27.825347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-e0b07894-e94a-48de-ab0a-8a6c7def4792 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:27.826613 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e0b07894-e94a-48de-ab0a-8a6c7def4792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:27.834688 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1330/2675] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:27.839043 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71871490-5c4d-4ae2-9189-d6eaba01273e None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:27.848892 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71871490-5c4d-4ae2-9189-d6eaba01273e None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:27.853698 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434da1ab-d040-46c3-9e16-028defe0aede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:27.854430 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434da1ab-d040-46c3-9e16-028defe0aede admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:27.854887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434da1ab-d040-46c3-9e16-028defe0aede admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:27.855038 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434da1ab-d040-46c3-9e16-028defe0aede admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:27.855761 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434da1ab-d040-46c3-9e16-028defe0aede admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:19:27.856067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434da1ab-d040-46c3-9e16-028defe0aede admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:27.857721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434da1ab-d040-46c3-9e16-028defe0aede admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:27.869732 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1346/2676] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:19:27.875205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b07894-e94a-48de-ab0a-8a6c7def4792 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:27.884842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b07894-e94a-48de-ab0a-8a6c7def4792 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:27.890447 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313bb7be-f257-4a33-865e-47419746d3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:27.891056 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313bb7be-f257-4a33-865e-47419746d3e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:27.891335 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313bb7be-f257-4a33-865e-47419746d3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:27.891608 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313bb7be-f257-4a33-865e-47419746d3e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:27.892280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313bb7be-f257-4a33-865e-47419746d3e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:27.894208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313bb7be-f257-4a33-865e-47419746d3e8 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:27.976168 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1331/2677] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:19:27 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:19:27.981125 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434da1ab-d040-46c3-9e16-028defe0aede admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:27.991323 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434da1ab-d040-46c3-9e16-028defe0aede admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:28.050421 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceacf7d-7287-4af8-bd31-1f49afe0456c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:28.051177 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceacf7d-7287-4af8-bd31-1f49afe0456c admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:28.051462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceacf7d-7287-4af8-bd31-1f49afe0456c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:28.051771 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceacf7d-7287-4af8-bd31-1f49afe0456c admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:28.052489 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceacf7d-7287-4af8-bd31-1f49afe0456c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:28.054483 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceacf7d-7287-4af8-bd31-1f49afe0456c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:28.152563 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1347/2678] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:19:27 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:19:28.157549 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313bb7be-f257-4a33-865e-47419746d3e8 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:28.169937 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313bb7be-f257-4a33-865e-47419746d3e8 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:28.174838 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fedbbf3-29be-4661-abbb-ed01f43925af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:28.175457 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fedbbf3-29be-4661-abbb-ed01f43925af admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:28.175744 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fedbbf3-29be-4661-abbb-ed01f43925af admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:28.176021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fedbbf3-29be-4661-abbb-ed01f43925af admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:28.176724 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fedbbf3-29be-4661-abbb-ed01f43925af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:28.178432 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fedbbf3-29be-4661-abbb-ed01f43925af admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:28.190718 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1332/2679] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:19:28.194683 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceacf7d-7287-4af8-bd31-1f49afe0456c admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:28.205022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceacf7d-7287-4af8-bd31-1f49afe0456c admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:28.210018 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9010a34c-ddc2-4853-9f36-3e0ac8103bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:28.212826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010a34c-ddc2-4853-9f36-3e0ac8103bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:28.212826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010a34c-ddc2-4853-9f36-3e0ac8103bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:28.212826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9010a34c-ddc2-4853-9f36-3e0ac8103bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:28.212826 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9010a34c-ddc2-4853-9f36-3e0ac8103bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:28.213845 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9010a34c-ddc2-4853-9f36-3e0ac8103bd4 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:28.225164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1348/2680] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:19:28.229333 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fedbbf3-29be-4661-abbb-ed01f43925af admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:28.239916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fedbbf3-29be-4661-abbb-ed01f43925af admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:28.244722 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddfc73-79dc-4efa-a738-3f7f245f0769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:28.245370 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddfc73-79dc-4efa-a738-3f7f245f0769 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:28.246240 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddfc73-79dc-4efa-a738-3f7f245f0769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:28.246529 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddfc73-79dc-4efa-a738-3f7f245f0769 admin admin] PATH_INFO: `/v3/projects/a5f946ccea9d497da7874e91a2064884/users/45a0c54a549e49a2a6987158a90ea669/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:28.261790 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddfc73-79dc-4efa-a738-3f7f245f0769 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5f946ccea9d497da7874e91a2064884, user_id=45a0c54a549e49a2a6987158a90ea669, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:28.264624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddfc73-79dc-4efa-a738-3f7f245f0769 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:28.350724 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1333/2681] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:28 2022] PUT /identity/v3/projects/a5f946ccea9d497da7874e91a2064884/users/45a0c54a549e49a2a6987158a90ea669/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:28.363894 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c3319-ed0f-4a1c-9ba5-589a76796c97 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:28.364237 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c3319-ed0f-4a1c-9ba5-589a76796c97 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:28.364527 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c3319-ed0f-4a1c-9ba5-589a76796c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:28.394157 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-708c3319-ed0f-4a1c-9ba5-589a76796c97 None None] MFA Rules not processed for user `45a0c54a549e49a2a6987158a90ea669`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:28.395408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-708c3319-ed0f-4a1c-9ba5-589a76796c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:28.446130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:28.466531 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1349/2682] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:28.471227 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddfc73-79dc-4efa-a738-3f7f245f0769 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:28.533940 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddfc73-79dc-4efa-a738-3f7f245f0769 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:28.538839 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33706290-a9b1-4f27-a585-584b08613f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:28.539432 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33706290-a9b1-4f27-a585-584b08613f29 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:28.539711 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33706290-a9b1-4f27-a585-584b08613f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:28.539980 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33706290-a9b1-4f27-a585-584b08613f29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:28.540634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33706290-a9b1-4f27-a585-584b08613f29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:28.543128 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33706290-a9b1-4f27-a585-584b08613f29 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:28.559116 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:28.672876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1334/2683] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:19:28 2022] POST /identity/v3/projects => generated 383 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:19:28.677148 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c3319-ed0f-4a1c-9ba5-589a76796c97 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:28.688252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c3319-ed0f-4a1c-9ba5-589a76796c97 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:28.747119 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd2802-d7ca-433c-a5f0-73dda95a5d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:28.747788 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd2802-d7ca-433c-a5f0-73dda95a5d27 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:28.747930 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd2802-d7ca-433c-a5f0-73dda95a5d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:28.748205 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd2802-d7ca-433c-a5f0-73dda95a5d27 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:28.748865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd2802-d7ca-433c-a5f0-73dda95a5d27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:28.750823 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd2802-d7ca-433c-a5f0-73dda95a5d27 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:28.843520 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1350/2684] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:19:28 2022] POST /identity/v3/users => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:19:28.847580 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33706290-a9b1-4f27-a585-584b08613f29 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:28.858612 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33706290-a9b1-4f27-a585-584b08613f29 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:28.863110 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b744d6-e742-439e-a254-851fa6278218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:28.863710 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b744d6-e742-439e-a254-851fa6278218 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:28.863988 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b744d6-e742-439e-a254-851fa6278218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:28.864260 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b744d6-e742-439e-a254-851fa6278218 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:28.864971 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b744d6-e742-439e-a254-851fa6278218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:28.866622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b744d6-e742-439e-a254-851fa6278218 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:28.878162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1335/2685] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:19:28.882246 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd2802-d7ca-433c-a5f0-73dda95a5d27 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:28.892435 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd2802-d7ca-433c-a5f0-73dda95a5d27 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:28.897329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97a2d3-b3cf-4c3d-a854-e616164b46e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:28.899922 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97a2d3-b3cf-4c3d-a854-e616164b46e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:28.899922 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97a2d3-b3cf-4c3d-a854-e616164b46e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:28.899922 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97a2d3-b3cf-4c3d-a854-e616164b46e0 admin admin] PATH_INFO: `/v3/projects/8e571bbae00a4ede832e5be23c1309d4/users/3ef5d70682d74119aa176d9f3b87d63e/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:28.913328 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97a2d3-b3cf-4c3d-a854-e616164b46e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e571bbae00a4ede832e5be23c1309d4, user_id=3ef5d70682d74119aa176d9f3b87d63e, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:28.916161 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97a2d3-b3cf-4c3d-a854-e616164b46e0 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:28.990389 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1351/2686] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:28 2022] PUT /identity/v3/projects/8e571bbae00a4ede832e5be23c1309d4/users/3ef5d70682d74119aa176d9f3b87d63e/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:28.994492 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b744d6-e742-439e-a254-851fa6278218 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:29.011377 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b744d6-e742-439e-a254-851fa6278218 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:29.069726 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4795602-fe0f-47e8-ada8-a31f2664a063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:29.070395 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4795602-fe0f-47e8-ada8-a31f2664a063 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:29.070681 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4795602-fe0f-47e8-ada8-a31f2664a063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:29.070959 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4795602-fe0f-47e8-ada8-a31f2664a063 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:29.071678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4795602-fe0f-47e8-ada8-a31f2664a063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:29.073281 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4795602-fe0f-47e8-ada8-a31f2664a063 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:29.084291 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1336/2687] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:19:29.088605 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97a2d3-b3cf-4c3d-a854-e616164b46e0 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:29.100029 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97a2d3-b3cf-4c3d-a854-e616164b46e0 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:29.105505 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545c640-adeb-4122-aea3-583c51fa0b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:29.106213 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545c640-adeb-4122-aea3-583c51fa0b59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:29.106495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545c640-adeb-4122-aea3-583c51fa0b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:29.106768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545c640-adeb-4122-aea3-583c51fa0b59 admin admin] PATH_INFO: `/v3/projects/8e571bbae00a4ede832e5be23c1309d4/users/3ef5d70682d74119aa176d9f3b87d63e/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:29.108715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545c640-adeb-4122-aea3-583c51fa0b59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e571bbae00a4ede832e5be23c1309d4, user_id=3ef5d70682d74119aa176d9f3b87d63e, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:29.111436 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e545c640-adeb-4122-aea3-583c51fa0b59 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:29.122534 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1352/2688] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:29 2022] PUT /identity/v3/projects/8e571bbae00a4ede832e5be23c1309d4/users/3ef5d70682d74119aa176d9f3b87d63e/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:19:29.133329 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd00e3a-fb2a-4b2a-b057-971fed8db52c None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:29.133656 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd00e3a-fb2a-4b2a-b057-971fed8db52c None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:29.133926 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd00e3a-fb2a-4b2a-b057-971fed8db52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:29.162294 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-ffd00e3a-fb2a-4b2a-b057-971fed8db52c None None] MFA Rules not processed for user `3ef5d70682d74119aa176d9f3b87d63e`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:29.163513 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-ffd00e3a-fb2a-4b2a-b057-971fed8db52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:29.233680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1337/2689] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:29 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:29.244660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545c640-adeb-4122-aea3-583c51fa0b59 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:29.246754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e545c640-adeb-4122-aea3-583c51fa0b59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:29.304922 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e545c640-adeb-4122-aea3-583c51fa0b59 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:29.309015 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70081a-e3d8-4458-a2f9-9a063b820fda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:29.309705 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70081a-e3d8-4458-a2f9-9a063b820fda service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:29.310044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70081a-e3d8-4458-a2f9-9a063b820fda service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:29.310204 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70081a-e3d8-4458-a2f9-9a063b820fda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:29.319921 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b70081a-e3d8-4458-a2f9-9a063b820fda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:29.321388 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b70081a-e3d8-4458-a2f9-9a063b820fda service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:29.331835 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1353/2690] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:29 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:19:29.439094 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd00e3a-fb2a-4b2a-b057-971fed8db52c None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:29.449930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd00e3a-fb2a-4b2a-b057-971fed8db52c None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:29.453660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb9a79-445e-443f-acce-646575dda913 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:29.454464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb9a79-445e-443f-acce-646575dda913 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:29.459435 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb9a79-445e-443f-acce-646575dda913 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:29.459435 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb9a79-445e-443f-acce-646575dda913 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:29.515931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb9a79-445e-443f-acce-646575dda913 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:29.517492 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb9a79-445e-443f-acce-646575dda913 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:29.528712 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1338/2691] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:19:30.094075 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70081a-e3d8-4458-a2f9-9a063b820fda service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:30.103310 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4b70081a-e3d8-4458-a2f9-9a063b820fda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:30.153353 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:30.173509 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70081a-e3d8-4458-a2f9-9a063b820fda service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:30.178131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0e3cc-0416-4885-a880-fb629aa2f1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:30.178768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0e3cc-0416-4885-a880-fb629aa2f1f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:30.179046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0e3cc-0416-4885-a880-fb629aa2f1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:30.179315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce0e3cc-0416-4885-a880-fb629aa2f1f8 admin admin] PATH_INFO: `/v3/users/74fcf1c35a2d496e93e0aa305277c6fc` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:30.180461 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce0e3cc-0416-4885-a880-fb629aa2f1f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74fcf1c35a2d496e93e0aa305277c6fc)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:30.182799 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce0e3cc-0416-4885-a880-fb629aa2f1f8 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:30.389739 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-dce0e3cc-0416-4885-a880-fb629aa2f1f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74fcf1c35a2d496e93e0aa305277c6fc'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:30.396968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-dce0e3cc-0416-4885-a880-fb629aa2f1f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74fcf1c35a2d496e93e0aa305277c6fc'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:30.397756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-dce0e3cc-0416-4885-a880-fb629aa2f1f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74fcf1c35a2d496e93e0aa305277c6fc'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:30.410950 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-dce0e3cc-0416-4885-a880-fb629aa2f1f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74fcf1c35a2d496e93e0aa305277c6fc'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:30.411714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-dce0e3cc-0416-4885-a880-fb629aa2f1f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74fcf1c35a2d496e93e0aa305277c6fc'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:30.471598 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1354/2692] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:19:30 2022] DELETE /identity/v3/users/74fcf1c35a2d496e93e0aa305277c6fc => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:30.476343 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d90aee-5bdb-41e1-86e9-9df8be7b36fe None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:30.476840 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d90aee-5bdb-41e1-86e9-9df8be7b36fe None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:30.476979 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d90aee-5bdb-41e1-86e9-9df8be7b36fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:30.486381 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:30.498187 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-f2d90aee-5bdb-41e1-86e9-9df8be7b36fe None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:30.499411 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f2d90aee-5bdb-41e1-86e9-9df8be7b36fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:30.568992 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1339/2693] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:30.579045 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0e3cc-0416-4885-a880-fb629aa2f1f8 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:30.580509 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-dce0e3cc-0416-4885-a880-fb629aa2f1f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:30.645066 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce0e3cc-0416-4885-a880-fb629aa2f1f8 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:30.648662 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907519f-442e-43bd-8307-700bc2125290 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:30.649283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907519f-442e-43bd-8307-700bc2125290 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:30.649565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907519f-442e-43bd-8307-700bc2125290 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:30.649834 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907519f-442e-43bd-8307-700bc2125290 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:30.663092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907519f-442e-43bd-8307-700bc2125290 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:30.664495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907519f-442e-43bd-8307-700bc2125290 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:30.676185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1355/2694] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:19:31.179918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d90aee-5bdb-41e1-86e9-9df8be7b36fe None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:31.185255 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-f2d90aee-5bdb-41e1-86e9-9df8be7b36fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:31.207931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d90aee-5bdb-41e1-86e9-9df8be7b36fe None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:31.222596 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f7447-cb43-4be0-8854-a96363bf54a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:31.223062 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f7447-cb43-4be0-8854-a96363bf54a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:31.223343 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f7447-cb43-4be0-8854-a96363bf54a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:31.223618 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f7447-cb43-4be0-8854-a96363bf54a4 admin admin] PATH_INFO: `/v3/projects/213d4ee9d8f34f13a532fde3f4ef252e` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:31.224921 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4f7447-cb43-4be0-8854-a96363bf54a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=213d4ee9d8f34f13a532fde3f4ef252e)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:31.227022 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4f7447-cb43-4be0-8854-a96363bf54a4 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:31.368065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-eb4f7447-cb43-4be0-8854-a96363bf54a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '213d4ee9d8f34f13a532fde3f4ef252e', 'request_id': 'req-eb4f7447-cb43-4be0-8854-a96363bf54a4'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:31.376301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-eb4f7447-cb43-4be0-8854-a96363bf54a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '213d4ee9d8f34f13a532fde3f4ef252e', 'request_id': 'req-eb4f7447-cb43-4be0-8854-a96363bf54a4'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:31.382243 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-eb4f7447-cb43-4be0-8854-a96363bf54a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '213d4ee9d8f34f13a532fde3f4ef252e', 'request_id': 'req-eb4f7447-cb43-4be0-8854-a96363bf54a4'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:31.477038 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-eb4f7447-cb43-4be0-8854-a96363bf54a4 admin admin] The token cache is being invalidate because project 213d4ee9d8f34f13a532fde3f4ef252e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:19:31.477756 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-eb4f7447-cb43-4be0-8854-a96363bf54a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:31.477861 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-eb4f7447-cb43-4be0-8854-a96363bf54a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:31.479689 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1340/2695] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:19:31 2022] DELETE /identity/v3/projects/213d4ee9d8f34f13a532fde3f4ef252e => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:31.483672 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907519f-442e-43bd-8307-700bc2125290 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:31.493790 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907519f-442e-43bd-8307-700bc2125290 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:31.561634 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ec015-9964-43cf-be0a-1d8f4aef4a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:31.562343 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ec015-9964-43cf-be0a-1d8f4aef4a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:31.562665 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ec015-9964-43cf-be0a-1d8f4aef4a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:31.562972 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ec015-9964-43cf-be0a-1d8f4aef4a64 admin admin] PATH_INFO: `/v3/users/2109a825c63e431a951b5cdb31b9ae23` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:31.564214 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ec015-9964-43cf-be0a-1d8f4aef4a64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2109a825c63e431a951b5cdb31b9ae23)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:31.566252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ec015-9964-43cf-be0a-1d8f4aef4a64 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:31.572344 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:31.789322 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d70ec015-9964-43cf-be0a-1d8f4aef4a64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2109a825c63e431a951b5cdb31b9ae23'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:31.796909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d70ec015-9964-43cf-be0a-1d8f4aef4a64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2109a825c63e431a951b5cdb31b9ae23'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:31.797715 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d70ec015-9964-43cf-be0a-1d8f4aef4a64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2109a825c63e431a951b5cdb31b9ae23'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:31.810937 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d70ec015-9964-43cf-be0a-1d8f4aef4a64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2109a825c63e431a951b5cdb31b9ae23'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:31.811700 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-d70ec015-9964-43cf-be0a-1d8f4aef4a64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2109a825c63e431a951b5cdb31b9ae23'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:31.878103 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1356/2696] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:19:31 2022] DELETE /identity/v3/users/2109a825c63e431a951b5cdb31b9ae23 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:32.341653 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f7447-cb43-4be0-8854-a96363bf54a4 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:32.343403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-eb4f7447-cb43-4be0-8854-a96363bf54a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:32.359089 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f7447-cb43-4be0-8854-a96363bf54a4 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:32.426821 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bad964-90d4-4543-acc1-34260cbc9f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:32.427466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bad964-90d4-4543-acc1-34260cbc9f07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:32.427743 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bad964-90d4-4543-acc1-34260cbc9f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:32.428017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bad964-90d4-4543-acc1-34260cbc9f07 admin admin] PATH_INFO: `/v3/projects/91e25fa6f3e344f2891ae245c92e481d` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:32.429186 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bad964-90d4-4543-acc1-34260cbc9f07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91e25fa6f3e344f2891ae245c92e481d)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:32.431366 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bad964-90d4-4543-acc1-34260cbc9f07 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:32.628760 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ec015-9964-43cf-be0a-1d8f4aef4a64 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:32.630239 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d70ec015-9964-43cf-be0a-1d8f4aef4a64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:32.644013 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ec015-9964-43cf-be0a-1d8f4aef4a64 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:32.648596 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8dadce-159c-4100-bac1-b56f24625e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:32.649233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8dadce-159c-4100-bac1-b56f24625e19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:32.649517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8dadce-159c-4100-bac1-b56f24625e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:32.649801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8dadce-159c-4100-bac1-b56f24625e19 admin admin] PATH_INFO: `/v3/users/45a0c54a549e49a2a6987158a90ea669` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:32.651004 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8dadce-159c-4100-bac1-b56f24625e19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45a0c54a549e49a2a6987158a90ea669)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:32.653517 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8dadce-159c-4100-bac1-b56f24625e19 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:32.659946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:32.672422 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-35bad964-90d4-4543-acc1-34260cbc9f07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91e25fa6f3e344f2891ae245c92e481d', 'request_id': 'req-35bad964-90d4-4543-acc1-34260cbc9f07'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:32.678633 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-35bad964-90d4-4543-acc1-34260cbc9f07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91e25fa6f3e344f2891ae245c92e481d', 'request_id': 'req-35bad964-90d4-4543-acc1-34260cbc9f07'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:32.684865 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-35bad964-90d4-4543-acc1-34260cbc9f07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91e25fa6f3e344f2891ae245c92e481d', 'request_id': 'req-35bad964-90d4-4543-acc1-34260cbc9f07'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:32.769259 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-35bad964-90d4-4543-acc1-34260cbc9f07 admin admin] The token cache is being invalidate because project 91e25fa6f3e344f2891ae245c92e481d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:19:32.769558 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-35bad964-90d4-4543-acc1-34260cbc9f07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:32.770213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-35bad964-90d4-4543-acc1-34260cbc9f07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:32.771743 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1341/2697] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:19:32 2022] DELETE /identity/v3/projects/91e25fa6f3e344f2891ae245c92e481d => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:32.778459 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7fcbe-eba6-4655-a06a-cee6940ee9c7 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:32.778777 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7fcbe-eba6-4655-a06a-cee6940ee9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:32.779059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7fcbe-eba6-4655-a06a-cee6940ee9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:32.798825 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-23f7fcbe-eba6-4655-a06a-cee6940ee9c7 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:32.800045 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-23f7fcbe-eba6-4655-a06a-cee6940ee9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:32.859442 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1342/2698] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:19:32.867514 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb74b9-9ab9-4eae-9bfb-4aa137ed2d8b None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:32.867807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb74b9-9ab9-4eae-9bfb-4aa137ed2d8b None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:32.868086 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb74b9-9ab9-4eae-9bfb-4aa137ed2d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:32.888450 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-cfdb74b9-9ab9-4eae-9bfb-4aa137ed2d8b None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:32.889668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-cfdb74b9-9ab9-4eae-9bfb-4aa137ed2d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:32.897805 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1343/2699] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:19:32.902148 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb74b9-9ab9-4eae-9bfb-4aa137ed2d8b None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:32.911076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb74b9-9ab9-4eae-9bfb-4aa137ed2d8b None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:32.915815 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbf657-9cf5-4a46-b313-ab15b5fb4739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:32.916526 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbf657-9cf5-4a46-b313-ab15b5fb4739 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:32.916824 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbf657-9cf5-4a46-b313-ab15b5fb4739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:32.917120 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadbf657-9cf5-4a46-b313-ab15b5fb4739 admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:32.918162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbf657-9cf5-4a46-b313-ab15b5fb4739 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:19:32.918433 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ec8dadce-159c-4100-bac1-b56f24625e19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45a0c54a549e49a2a6987158a90ea669'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:32.918566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbf657-9cf5-4a46-b313-ab15b5fb4739 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:32.921126 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadbf657-9cf5-4a46-b313-ab15b5fb4739 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:32.927374 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ec8dadce-159c-4100-bac1-b56f24625e19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45a0c54a549e49a2a6987158a90ea669'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:32.928120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ec8dadce-159c-4100-bac1-b56f24625e19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45a0c54a549e49a2a6987158a90ea669'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:32.939611 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1344/2700] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:19:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:19:32.944049 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbf657-9cf5-4a46-b313-ab15b5fb4739 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:32.951666 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ec8dadce-159c-4100-bac1-b56f24625e19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45a0c54a549e49a2a6987158a90ea669'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:32.952390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-ec8dadce-159c-4100-bac1-b56f24625e19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45a0c54a549e49a2a6987158a90ea669'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:32.956754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadbf657-9cf5-4a46-b313-ab15b5fb4739 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:32.961670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd3cff-4cef-4d18-bc88-84c49e0f08a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:32.962328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd3cff-4cef-4d18-bc88-84c49e0f08a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:32.962616 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd3cff-4cef-4d18-bc88-84c49e0f08a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:32.962896 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd3cff-4cef-4d18-bc88-84c49e0f08a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:32.963568 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd3cff-4cef-4d18-bc88-84c49e0f08a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:32.965548 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd3cff-4cef-4d18-bc88-84c49e0f08a8 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:33.030337 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1357/2701] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:19:32 2022] DELETE /identity/v3/users/45a0c54a549e49a2a6987158a90ea669 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:33.034887 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b57e1f-5eb0-4a3a-9a36-110a798dc1e5 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:33.035375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b57e1f-5eb0-4a3a-9a36-110a798dc1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:33.035888 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b57e1f-5eb0-4a3a-9a36-110a798dc1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:33.055271 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-58b57e1f-5eb0-4a3a-9a36-110a798dc1e5 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:33.056806 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-58b57e1f-5eb0-4a3a-9a36-110a798dc1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:33.064715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1345/2702] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:19:32 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:19:33.069115 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd3cff-4cef-4d18-bc88-84c49e0f08a8 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:33.070968 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-b2bd3cff-4cef-4d18-bc88-84c49e0f08a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:33.111381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:33.126424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1358/2703] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:19:33.138219 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b57e1f-5eb0-4a3a-9a36-110a798dc1e5 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:33.139597 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-58b57e1f-5eb0-4a3a-9a36-110a798dc1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:33.141835 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd3cff-4cef-4d18-bc88-84c49e0f08a8 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:33.147185 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602f34e-27e6-42df-935a-55e68a27b449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:33.147815 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602f34e-27e6-42df-935a-55e68a27b449 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:33.148101 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602f34e-27e6-42df-935a-55e68a27b449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:33.148379 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1602f34e-27e6-42df-935a-55e68a27b449 admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:33.149052 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602f34e-27e6-42df-935a-55e68a27b449 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:33.151278 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1602f34e-27e6-42df-935a-55e68a27b449 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:33.198130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b57e1f-5eb0-4a3a-9a36-110a798dc1e5 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:33.241802 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1346/2704] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:19:33 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:19:33.246093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602f34e-27e6-42df-935a-55e68a27b449 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:33.252803 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb1196-96c6-4f7f-b58a-6ef97f71858f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:33.254494 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb1196-96c6-4f7f-b58a-6ef97f71858f service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:33.254773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb1196-96c6-4f7f-b58a-6ef97f71858f service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:33.255044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eb1196-96c6-4f7f-b58a-6ef97f71858f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:33.256182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1602f34e-27e6-42df-935a-55e68a27b449 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:33.262055 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68666c4f-5e67-4ac4-867b-c4df4ddd5b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:33.262707 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68666c4f-5e67-4ac4-867b-c4df4ddd5b95 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:33.262995 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68666c4f-5e67-4ac4-867b-c4df4ddd5b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:33.263278 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68666c4f-5e67-4ac4-867b-c4df4ddd5b95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:33.263986 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68666c4f-5e67-4ac4-867b-c4df4ddd5b95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:33.265359 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb1196-96c6-4f7f-b58a-6ef97f71858f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:33.265598 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68666c4f-5e67-4ac4-867b-c4df4ddd5b95 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:33.267008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7eb1196-96c6-4f7f-b58a-6ef97f71858f service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:33.267132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:33.277566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1347/2705] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:19:33.278388 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1359/2706] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:19:33.282341 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68666c4f-5e67-4ac4-867b-c4df4ddd5b95 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:33.292745 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68666c4f-5e67-4ac4-867b-c4df4ddd5b95 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:33.297639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb26986-7285-47f1-aa4a-802e1d448249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:33.298333 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb26986-7285-47f1-aa4a-802e1d448249 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:33.298643 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb26986-7285-47f1-aa4a-802e1d448249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:33.298935 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb26986-7285-47f1-aa4a-802e1d448249 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:33.299607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb26986-7285-47f1-aa4a-802e1d448249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:33.301313 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb26986-7285-47f1-aa4a-802e1d448249 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:33.313915 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1348/2707] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:19:33.318157 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb1196-96c6-4f7f-b58a-6ef97f71858f service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:33.329005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eb1196-96c6-4f7f-b58a-6ef97f71858f service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:33.334097 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1387eb-f258-4a83-b07f-82c2c3c2200f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:33.334997 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1387eb-f258-4a83-b07f-82c2c3c2200f admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:33.335278 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1387eb-f258-4a83-b07f-82c2c3c2200f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:33.335560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1387eb-f258-4a83-b07f-82c2c3c2200f admin admin] PATH_INFO: `/v3/projects/012a8fb9ecc940edb934c78533d603c2/users/e49214d9458b4715a6c05a5cf7fdb190/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:33.351964 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1387eb-f258-4a83-b07f-82c2c3c2200f admin admin] RBAC: Authorizing `identity:create_grant(project_id=012a8fb9ecc940edb934c78533d603c2, user_id=e49214d9458b4715a6c05a5cf7fdb190, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:33.354743 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1387eb-f258-4a83-b07f-82c2c3c2200f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:33.461283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1360/2708] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:33 2022] PUT /identity/v3/projects/012a8fb9ecc940edb934c78533d603c2/users/e49214d9458b4715a6c05a5cf7fdb190/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:33.474878 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a6599-fe75-4632-a353-c0c7599cd683 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:33.475168 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a6599-fe75-4632-a353-c0c7599cd683 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:33.475508 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6a6599-fe75-4632-a353-c0c7599cd683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:33.511512 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-dc6a6599-fe75-4632-a353-c0c7599cd683 None None] MFA Rules not processed for user `e49214d9458b4715a6c05a5cf7fdb190`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:33.512449 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-dc6a6599-fe75-4632-a353-c0c7599cd683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:33.582816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1349/2709] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:33.587721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1387eb-f258-4a83-b07f-82c2c3c2200f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:33.654429 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1387eb-f258-4a83-b07f-82c2c3c2200f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:33.661752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f8ff3-8aec-48e9-b86e-0cf827c56ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:33.662401 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f8ff3-8aec-48e9-b86e-0cf827c56ea5 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:33.662710 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f8ff3-8aec-48e9-b86e-0cf827c56ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:33.662982 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f8ff3-8aec-48e9-b86e-0cf827c56ea5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:33.663658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f8ff3-8aec-48e9-b86e-0cf827c56ea5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:33.665556 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f8ff3-8aec-48e9-b86e-0cf827c56ea5 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:33.752745 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a6599-fe75-4632-a353-c0c7599cd683 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:33.754284 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-dc6a6599-fe75-4632-a353-c0c7599cd683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:33.769117 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6a6599-fe75-4632-a353-c0c7599cd683 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:33.775328 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc45bdc1-0184-4628-9d08-f9b49b938758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:33.775972 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45bdc1-0184-4628-9d08-f9b49b938758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:33.779357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45bdc1-0184-4628-9d08-f9b49b938758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:33.779357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45bdc1-0184-4628-9d08-f9b49b938758 admin admin] PATH_INFO: `/v3/projects/a5f946ccea9d497da7874e91a2064884` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:33.779357 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc45bdc1-0184-4628-9d08-f9b49b938758 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5f946ccea9d497da7874e91a2064884)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:33.779744 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc45bdc1-0184-4628-9d08-f9b49b938758 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:33.787069 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1361/2710] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:19:33 2022] POST /identity/v3/projects => generated 383 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:19:33.787745 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f8ff3-8aec-48e9-b86e-0cf827c56ea5 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:33.806658 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:33.858039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f8ff3-8aec-48e9-b86e-0cf827c56ea5 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:33.862979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06db818-6348-432c-9a55-80ae85f7f202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:33.863602 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06db818-6348-432c-9a55-80ae85f7f202 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:33.863888 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06db818-6348-432c-9a55-80ae85f7f202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:33.864168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06db818-6348-432c-9a55-80ae85f7f202 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:33.864840 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06db818-6348-432c-9a55-80ae85f7f202 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:33.866886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06db818-6348-432c-9a55-80ae85f7f202 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:33.884598 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-bc45bdc1-0184-4628-9d08-f9b49b938758 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5f946ccea9d497da7874e91a2064884', 'request_id': 'req-bc45bdc1-0184-4628-9d08-f9b49b938758'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:33.891676 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-bc45bdc1-0184-4628-9d08-f9b49b938758 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5f946ccea9d497da7874e91a2064884', 'request_id': 'req-bc45bdc1-0184-4628-9d08-f9b49b938758'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:33.897590 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-bc45bdc1-0184-4628-9d08-f9b49b938758 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5f946ccea9d497da7874e91a2064884', 'request_id': 'req-bc45bdc1-0184-4628-9d08-f9b49b938758'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:33.980729 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-bc45bdc1-0184-4628-9d08-f9b49b938758 admin admin] The token cache is being invalidate because project a5f946ccea9d497da7874e91a2064884 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:19:33.981021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-bc45bdc1-0184-4628-9d08-f9b49b938758 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:33.981558 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1362/2711] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:19:33 2022] POST /identity/v3/users => generated 292 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:19:33.981681 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-bc45bdc1-0184-4628-9d08-f9b49b938758 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:33.983242 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1350/2712] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:19:33 2022] DELETE /identity/v3/projects/a5f946ccea9d497da7874e91a2064884 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:33.985704 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06db818-6348-432c-9a55-80ae85f7f202 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:33.986908 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc45bdc1-0184-4628-9d08-f9b49b938758 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:33.987623 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a06db818-6348-432c-9a55-80ae85f7f202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:33.988215 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-bc45bdc1-0184-4628-9d08-f9b49b938758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:34.000385 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06db818-6348-432c-9a55-80ae85f7f202 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:34.052385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:34.056030 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc45bdc1-0184-4628-9d08-f9b49b938758 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:34.059031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2856be-6302-4e0e-ab45-cee26c2475d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:34.059664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2856be-6302-4e0e-ab45-cee26c2475d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:34.059970 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2856be-6302-4e0e-ab45-cee26c2475d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:34.060253 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2856be-6302-4e0e-ab45-cee26c2475d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:34.060988 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2856be-6302-4e0e-ab45-cee26c2475d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:34.061948 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dea37f-86b7-41df-86a7-3f34d56806cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:34.062588 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dea37f-86b7-41df-86a7-3f34d56806cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:34.062698 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2856be-6302-4e0e-ab45-cee26c2475d4 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:34.062869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dea37f-86b7-41df-86a7-3f34d56806cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:34.063141 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dea37f-86b7-41df-86a7-3f34d56806cb admin admin] PATH_INFO: `/v3/users/3ef5d70682d74119aa176d9f3b87d63e` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:34.064312 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dea37f-86b7-41df-86a7-3f34d56806cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ef5d70682d74119aa176d9f3b87d63e)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:34.066241 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dea37f-86b7-41df-86a7-3f34d56806cb admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:34.073170 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1363/2713] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:33 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:19:34.077162 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2856be-6302-4e0e-ab45-cee26c2475d4 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:34.087282 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2856be-6302-4e0e-ab45-cee26c2475d4 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:34.092214 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940573e4-737e-45c9-bc08-d24a33085c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:34.092838 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940573e4-737e-45c9-bc08-d24a33085c81 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:34.093128 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940573e4-737e-45c9-bc08-d24a33085c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:34.093411 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940573e4-737e-45c9-bc08-d24a33085c81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:34.094149 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940573e4-737e-45c9-bc08-d24a33085c81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:34.095765 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940573e4-737e-45c9-bc08-d24a33085c81 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:34.107603 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1364/2714] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:19:34.111812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940573e4-737e-45c9-bc08-d24a33085c81 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:34.123047 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940573e4-737e-45c9-bc08-d24a33085c81 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:34.127726 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b7681-3c44-42b7-a2be-378391546ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:34.128393 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b7681-3c44-42b7-a2be-378391546ad7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:34.128680 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b7681-3c44-42b7-a2be-378391546ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:34.128961 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b7681-3c44-42b7-a2be-378391546ad7 admin admin] PATH_INFO: `/v3/projects/2c0ac06be2734275872d94bb2760a1e0/users/4c620a476663412791ac70732d1e1741/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:34.144211 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b7681-3c44-42b7-a2be-378391546ad7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c0ac06be2734275872d94bb2760a1e0, user_id=4c620a476663412791ac70732d1e1741, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:34.147050 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b7681-3c44-42b7-a2be-378391546ad7 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:34.226190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1365/2715] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:34 2022] PUT /identity/v3/projects/2c0ac06be2734275872d94bb2760a1e0/users/4c620a476663412791ac70732d1e1741/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:19:34.236988 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204e99a-4f3f-45e3-820f-0877b0ea6923 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:34.237281 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204e99a-4f3f-45e3-820f-0877b0ea6923 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:34.237568 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204e99a-4f3f-45e3-820f-0877b0ea6923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:34.257711 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-07dea37f-86b7-41df-86a7-3f34d56806cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ef5d70682d74119aa176d9f3b87d63e'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:34.266454 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-07dea37f-86b7-41df-86a7-3f34d56806cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ef5d70682d74119aa176d9f3b87d63e'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:34.267087 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f204e99a-4f3f-45e3-820f-0877b0ea6923 None None] MFA Rules not processed for user `4c620a476663412791ac70732d1e1741`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:34.267213 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-07dea37f-86b7-41df-86a7-3f34d56806cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ef5d70682d74119aa176d9f3b87d63e'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:34.268537 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f204e99a-4f3f-45e3-820f-0877b0ea6923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:34.325574 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:34.334006 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-07dea37f-86b7-41df-86a7-3f34d56806cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ef5d70682d74119aa176d9f3b87d63e'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:34.334295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1366/2716] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:19:34.339529 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204e99a-4f3f-45e3-820f-0877b0ea6923 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:34.348850 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-07dea37f-86b7-41df-86a7-3f34d56806cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ef5d70682d74119aa176d9f3b87d63e'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:34.355962 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1351/2717] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:19:33 2022] DELETE /identity/v3/users/3ef5d70682d74119aa176d9f3b87d63e => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:34.403349 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204e99a-4f3f-45e3-820f-0877b0ea6923 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:34.466858 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7265f-72d7-4f6c-9c4a-8bfd61f50cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:34.467490 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7265f-72d7-4f6c-9c4a-8bfd61f50cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:34.467776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7265f-72d7-4f6c-9c4a-8bfd61f50cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:34.468056 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e7265f-72d7-4f6c-9c4a-8bfd61f50cf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:34.468736 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e7265f-72d7-4f6c-9c4a-8bfd61f50cf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:34.470703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e7265f-72d7-4f6c-9c4a-8bfd61f50cf5 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:34.569367 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1367/2718] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:19:34 2022] POST /identity/v3/projects => generated 385 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:19:34.573464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dea37f-86b7-41df-86a7-3f34d56806cb admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:34.575406 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-07dea37f-86b7-41df-86a7-3f34d56806cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:34.590881 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dea37f-86b7-41df-86a7-3f34d56806cb admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:34.652366 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f8d60-3aa7-4854-976a-175940c038ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:34.653035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f8d60-3aa7-4854-976a-175940c038ea admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:34.653321 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f8d60-3aa7-4854-976a-175940c038ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:34.653596 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6f8d60-3aa7-4854-976a-175940c038ea admin admin] PATH_INFO: `/v3/users` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:34.654323 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f8d60-3aa7-4854-976a-175940c038ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:34.656251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6f8d60-3aa7-4854-976a-175940c038ea admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:34.825163 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1352/2719] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:19:34 2022] POST /identity/v3/users => generated 299 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:19:34.829348 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7265f-72d7-4f6c-9c4a-8bfd61f50cf5 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:34.840468 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e7265f-72d7-4f6c-9c4a-8bfd61f50cf5 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:34.845523 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0e9d2-8cf5-4217-8cbd-479dbd67cbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:34.848423 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0e9d2-8cf5-4217-8cbd-479dbd67cbbd admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:34.848423 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0e9d2-8cf5-4217-8cbd-479dbd67cbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:34.848423 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0e9d2-8cf5-4217-8cbd-479dbd67cbbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:34.848423 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0e9d2-8cf5-4217-8cbd-479dbd67cbbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:34.849065 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0e9d2-8cf5-4217-8cbd-479dbd67cbbd admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:34.851306 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f8d60-3aa7-4854-976a-175940c038ea admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:34.853130 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-9e6f8d60-3aa7-4854-976a-175940c038ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:34.862814 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1368/2720] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:19:34.867563 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0e9d2-8cf5-4217-8cbd-479dbd67cbbd admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:34.868740 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6f8d60-3aa7-4854-976a-175940c038ea admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:34.873524 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a98a3b-54b5-44cc-b539-7d72e440dee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:34.874225 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a98a3b-54b5-44cc-b539-7d72e440dee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:34.874491 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a98a3b-54b5-44cc-b539-7d72e440dee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:34.874765 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a98a3b-54b5-44cc-b539-7d72e440dee6 admin admin] PATH_INFO: `/v3/projects/8e571bbae00a4ede832e5be23c1309d4` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:34.876236 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a98a3b-54b5-44cc-b539-7d72e440dee6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e571bbae00a4ede832e5be23c1309d4)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:34.877869 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a98a3b-54b5-44cc-b539-7d72e440dee6 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:34.879408 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0e9d2-8cf5-4217-8cbd-479dbd67cbbd admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:34.886383 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2d00d-8b89-4f23-81a8-4d795eca3738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:34.886383 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2d00d-8b89-4f23-81a8-4d795eca3738 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:34.886383 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2d00d-8b89-4f23-81a8-4d795eca3738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:34.886383 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2d00d-8b89-4f23-81a8-4d795eca3738 admin admin] PATH_INFO: `/v3/projects/fd0c006a2d534dbf82fea5a4d86f571b/users/23f55fbd296c4851aa27fdf78e039588/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:34.987103 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-f8a98a3b-54b5-44cc-b539-7d72e440dee6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e571bbae00a4ede832e5be23c1309d4', 'request_id': 'req-f8a98a3b-54b5-44cc-b539-7d72e440dee6'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:34.995591 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-f8a98a3b-54b5-44cc-b539-7d72e440dee6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e571bbae00a4ede832e5be23c1309d4', 'request_id': 'req-f8a98a3b-54b5-44cc-b539-7d72e440dee6'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:35.001006 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=80,keystone:DELETE=16,keystone:INSERT=4 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:35.002344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-f8a98a3b-54b5-44cc-b539-7d72e440dee6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e571bbae00a4ede832e5be23c1309d4', 'request_id': 'req-f8a98a3b-54b5-44cc-b539-7d72e440dee6'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:35.028371 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de2d00d-8b89-4f23-81a8-4d795eca3738 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd0c006a2d534dbf82fea5a4d86f571b, user_id=23f55fbd296c4851aa27fdf78e039588, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:35.031181 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de2d00d-8b89-4f23-81a8-4d795eca3738 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:35.112459 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-f8a98a3b-54b5-44cc-b539-7d72e440dee6 admin admin] The token cache is being invalidate because project 8e571bbae00a4ede832e5be23c1309d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:19:35.112752 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-f8a98a3b-54b5-44cc-b539-7d72e440dee6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:35.119004 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-f8a98a3b-54b5-44cc-b539-7d72e440dee6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:35.119004 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1353/2721] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:19:34 2022] DELETE /identity/v3/projects/8e571bbae00a4ede832e5be23c1309d4 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:19:35.119717 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af9852-efe3-4d2a-8844-c8b661786738 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:35.120006 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af9852-efe3-4d2a-8844-c8b661786738 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:35.120286 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af9852-efe3-4d2a-8844-c8b661786738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:35.140074 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-e0af9852-efe3-4d2a-8844-c8b661786738 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:35.142066 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e0af9852-efe3-4d2a-8844-c8b661786738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:35.146263 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1369/2722] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:34 2022] PUT /identity/v3/projects/fd0c006a2d534dbf82fea5a4d86f571b/users/23f55fbd296c4851aa27fdf78e039588/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:35.153928 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2d00d-8b89-4f23-81a8-4d795eca3738 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:35.155777 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0de2d00d-8b89-4f23-81a8-4d795eca3738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:35.204828 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1354/2723] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:19:35.207940 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:35.212651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49aab8-4121-4261-a6c3-946e72919138 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:35.213022 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49aab8-4121-4261-a6c3-946e72919138 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:35.213464 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49aab8-4121-4261-a6c3-946e72919138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:35.220884 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2d00d-8b89-4f23-81a8-4d795eca3738 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:35.225520 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd559ffa-4c0b-4f74-86e3-eecc4fcd3df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:35.226180 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd559ffa-4c0b-4f74-86e3-eecc4fcd3df7 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:35.226462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd559ffa-4c0b-4f74-86e3-eecc4fcd3df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:35.226737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd559ffa-4c0b-4f74-86e3-eecc4fcd3df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:35.227434 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd559ffa-4c0b-4f74-86e3-eecc4fcd3df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:35.229015 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd559ffa-4c0b-4f74-86e3-eecc4fcd3df7 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:35.234762 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-da49aab8-4121-4261-a6c3-946e72919138 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:35.236231 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-da49aab8-4121-4261-a6c3-946e72919138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:35.242604 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1370/2724] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:19:35.244813 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1355/2725] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:19:35.246971 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd559ffa-4c0b-4f74-86e3-eecc4fcd3df7 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:35.249097 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49aab8-4121-4261-a6c3-946e72919138 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:35.257150 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd559ffa-4c0b-4f74-86e3-eecc4fcd3df7 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:35.258644 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49aab8-4121-4261-a6c3-946e72919138 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:35.262703 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72320a3e-9a8e-4f10-a982-66f8b17f5f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:35.263372 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72320a3e-9a8e-4f10-a982-66f8b17f5f31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:35.263679 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72320a3e-9a8e-4f10-a982-66f8b17f5f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:35.263975 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72320a3e-9a8e-4f10-a982-66f8b17f5f31 admin admin] PATH_INFO: `/v3/projects/fd0c006a2d534dbf82fea5a4d86f571b/users/23f55fbd296c4851aa27fdf78e039588/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:35.264822 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68137c55-4158-40fc-b3c9-df231f93491a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:35.265442 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68137c55-4158-40fc-b3c9-df231f93491a admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:35.265726 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68137c55-4158-40fc-b3c9-df231f93491a admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:35.265997 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72320a3e-9a8e-4f10-a982-66f8b17f5f31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd0c006a2d534dbf82fea5a4d86f571b, user_id=23f55fbd296c4851aa27fdf78e039588, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:35.266131 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68137c55-4158-40fc-b3c9-df231f93491a admin admin] PATH_INFO: `/v3/domains` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:35.266797 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68137c55-4158-40fc-b3c9-df231f93491a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80329) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:19:35.267133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68137c55-4158-40fc-b3c9-df231f93491a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:35.268714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72320a3e-9a8e-4f10-a982-66f8b17f5f31 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:35.268857 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68137c55-4158-40fc-b3c9-df231f93491a admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:35.279535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1371/2726] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:35 2022] PUT /identity/v3/projects/fd0c006a2d534dbf82fea5a4d86f571b/users/23f55fbd296c4851aa27fdf78e039588/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:19:35.281255 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1356/2727] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:19:35.285347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72320a3e-9a8e-4f10-a982-66f8b17f5f31 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:35.295166 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d5f11-2484-463a-b3d1-899d52bd20a0 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:35.295462 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d5f11-2484-463a-b3d1-899d52bd20a0 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:35.295748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d5f11-2484-463a-b3d1-899d52bd20a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:35.325658 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-c86d5f11-2484-463a-b3d1-899d52bd20a0 None None] MFA Rules not processed for user `23f55fbd296c4851aa27fdf78e039588`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:35.326936 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-c86d5f11-2484-463a-b3d1-899d52bd20a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:35.350307 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72320a3e-9a8e-4f10-a982-66f8b17f5f31 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:35.355214 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fd578-9cd0-45e6-9a9e-96855e331dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:35.355840 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fd578-9cd0-45e6-9a9e-96855e331dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:35.356126 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fd578-9cd0-45e6-9a9e-96855e331dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:35.356402 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fd578-9cd0-45e6-9a9e-96855e331dd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:35.357091 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9fd578-9cd0-45e6-9a9e-96855e331dd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:35.359040 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9fd578-9cd0-45e6-9a9e-96855e331dd0 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:35.397332 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1357/2728] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:35.409278 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d5f11-2484-463a-b3d1-899d52bd20a0 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:35.410647 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-c86d5f11-2484-463a-b3d1-899d52bd20a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:35.464903 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1372/2729] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:19:35 2022] POST /identity/v3/projects => generated 383 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:19:35.466363 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fd578-9cd0-45e6-9a9e-96855e331dd0 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:35.469280 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d5f11-2484-463a-b3d1-899d52bd20a0 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:35.479071 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:35.521145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25605151-3fc7-4829-87b8-0a9dd9861efd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:35.521794 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25605151-3fc7-4829-87b8-0a9dd9861efd service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:35.522104 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25605151-3fc7-4829-87b8-0a9dd9861efd service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:35.522381 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25605151-3fc7-4829-87b8-0a9dd9861efd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:35.524216 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-25605151-3fc7-4829-87b8-0a9dd9861efd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:35.528516 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fd578-9cd0-45e6-9a9e-96855e331dd0 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:35.533378 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4211a-e4f2-4a8c-922a-ba684df421b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:35.534021 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4211a-e4f2-4a8c-922a-ba684df421b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:35.534304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4211a-e4f2-4a8c-922a-ba684df421b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:35.534578 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4211a-e4f2-4a8c-922a-ba684df421b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:35.535234 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e4211a-e4f2-4a8c-922a-ba684df421b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:35.537129 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e4211a-e4f2-4a8c-922a-ba684df421b0 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:35.587645 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25605151-3fc7-4829-87b8-0a9dd9861efd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:35.589143 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25605151-3fc7-4829-87b8-0a9dd9861efd service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:35.597986 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1358/2730] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 09:19:35.631448 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1373/2731] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:19:35 2022] POST /identity/v3/users => generated 292 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:19:35.638032 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25605151-3fc7-4829-87b8-0a9dd9861efd service cinder] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:35.647880 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25605151-3fc7-4829-87b8-0a9dd9861efd service cinder] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:35.652919 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313516a0-fe1c-49a1-ad63-7b1004fb6173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:35.653671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313516a0-fe1c-49a1-ad63-7b1004fb6173 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:35.654214 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313516a0-fe1c-49a1-ad63-7b1004fb6173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:35.654614 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313516a0-fe1c-49a1-ad63-7b1004fb6173 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:35.655410 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313516a0-fe1c-49a1-ad63-7b1004fb6173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:35.657105 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313516a0-fe1c-49a1-ad63-7b1004fb6173 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:35.668369 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1359/2732] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:19:35.672385 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4211a-e4f2-4a8c-922a-ba684df421b0 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:35.682591 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4211a-e4f2-4a8c-922a-ba684df421b0 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:35.688087 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1038d9-4512-479f-987b-e27b44574c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:35.688782 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1038d9-4512-479f-987b-e27b44574c5e admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:35.689067 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1038d9-4512-479f-987b-e27b44574c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:35.689343 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1038d9-4512-479f-987b-e27b44574c5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:35.690076 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1038d9-4512-479f-987b-e27b44574c5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:35.691720 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1038d9-4512-479f-987b-e27b44574c5e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:35.702991 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1374/2733] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:19:35.707329 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313516a0-fe1c-49a1-ad63-7b1004fb6173 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:35.717599 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313516a0-fe1c-49a1-ad63-7b1004fb6173 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:35.723254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bbaf6-19cd-434f-94c7-76bfeccd0af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:35.723930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bbaf6-19cd-434f-94c7-76bfeccd0af9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:35.724351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bbaf6-19cd-434f-94c7-76bfeccd0af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:35.724637 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bbaf6-19cd-434f-94c7-76bfeccd0af9 admin admin] PATH_INFO: `/v3/projects/4686c7a4d6d844bfa8bb94ac7f814d55/users/6bbde9f16c0e45b7b3515380e608d9af/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:35.742062 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145bbaf6-19cd-434f-94c7-76bfeccd0af9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4686c7a4d6d844bfa8bb94ac7f814d55, user_id=6bbde9f16c0e45b7b3515380e608d9af, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:35.744750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145bbaf6-19cd-434f-94c7-76bfeccd0af9 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:35.854364 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1360/2734] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:35 2022] PUT /identity/v3/projects/4686c7a4d6d844bfa8bb94ac7f814d55/users/6bbde9f16c0e45b7b3515380e608d9af/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:35.868798 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f520a7b-c422-4a11-8be4-e5c986304294 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:35.869306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f520a7b-c422-4a11-8be4-e5c986304294 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:35.869306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f520a7b-c422-4a11-8be4-e5c986304294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:35.897266 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-8f520a7b-c422-4a11-8be4-e5c986304294 None None] MFA Rules not processed for user `6bbde9f16c0e45b7b3515380e608d9af`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:35.898538 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-8f520a7b-c422-4a11-8be4-e5c986304294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:35.929088 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:35.964981 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1375/2735] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:35.969668 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bbaf6-19cd-434f-94c7-76bfeccd0af9 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:36.035267 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bbaf6-19cd-434f-94c7-76bfeccd0af9 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:36.040181 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723129dc-f4b4-4fb3-bd13-a6afc5ba5e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:36.040796 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723129dc-f4b4-4fb3-bd13-a6afc5ba5e6b admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:36.041076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723129dc-f4b4-4fb3-bd13-a6afc5ba5e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:36.041393 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723129dc-f4b4-4fb3-bd13-a6afc5ba5e6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:36.042645 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723129dc-f4b4-4fb3-bd13-a6afc5ba5e6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:36.044544 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723129dc-f4b4-4fb3-bd13-a6afc5ba5e6b admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:36.158919 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1361/2736] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:19:35 2022] POST /identity/v3/projects => generated 381 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:19:36.163027 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f520a7b-c422-4a11-8be4-e5c986304294 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:36.173809 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f520a7b-c422-4a11-8be4-e5c986304294 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:36.241259 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6737f-e62c-4d66-b88f-9442b90a3eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:36.241956 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6737f-e62c-4d66-b88f-9442b90a3eaa admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:36.242269 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6737f-e62c-4d66-b88f-9442b90a3eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:36.242543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6737f-e62c-4d66-b88f-9442b90a3eaa admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:36.243234 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6737f-e62c-4d66-b88f-9442b90a3eaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:36.245735 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6737f-e62c-4d66-b88f-9442b90a3eaa admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:36.340181 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1376/2737] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:19:36 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:19:36.344572 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723129dc-f4b4-4fb3-bd13-a6afc5ba5e6b admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:36.355711 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723129dc-f4b4-4fb3-bd13-a6afc5ba5e6b admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:36.360315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917328d-a317-4edf-a800-34b89870ead6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:36.360922 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917328d-a317-4edf-a800-34b89870ead6 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:36.361207 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917328d-a317-4edf-a800-34b89870ead6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:36.361485 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917328d-a317-4edf-a800-34b89870ead6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:36.362206 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917328d-a317-4edf-a800-34b89870ead6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:36.363809 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917328d-a317-4edf-a800-34b89870ead6 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:36.375350 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1362/2738] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:19:36.379769 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6737f-e62c-4d66-b88f-9442b90a3eaa admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:36.390830 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6737f-e62c-4d66-b88f-9442b90a3eaa admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:36.395564 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a6906-3977-46e2-a72b-68f03899f495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:36.396229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a6906-3977-46e2-a72b-68f03899f495 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:36.398048 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a6906-3977-46e2-a72b-68f03899f495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:36.398048 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a6906-3977-46e2-a72b-68f03899f495 admin admin] PATH_INFO: `/v3/projects/60521ccbb32e481e95a33085b4a52adc/users/164b2c2fa1e841f7a0b8896afbde4bb3/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:36.411643 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a6906-3977-46e2-a72b-68f03899f495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60521ccbb32e481e95a33085b4a52adc, user_id=164b2c2fa1e841f7a0b8896afbde4bb3, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:36.414474 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a6906-3977-46e2-a72b-68f03899f495 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:36.477008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1377/2739] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:36 2022] PUT /identity/v3/projects/60521ccbb32e481e95a33085b4a52adc/users/164b2c2fa1e841f7a0b8896afbde4bb3/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:36.482497 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917328d-a317-4edf-a800-34b89870ead6 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:36.488931 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:36.492667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917328d-a317-4edf-a800-34b89870ead6 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:36.549858 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af757d-0f42-4023-92eb-f7f94981b6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:36.550512 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af757d-0f42-4023-92eb-f7f94981b6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:36.550793 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af757d-0f42-4023-92eb-f7f94981b6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:36.551066 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af757d-0f42-4023-92eb-f7f94981b6b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:36.551755 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af757d-0f42-4023-92eb-f7f94981b6b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:36.553348 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af757d-0f42-4023-92eb-f7f94981b6b7 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:36.563673 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1363/2740] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:19:36.567865 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a6906-3977-46e2-a72b-68f03899f495 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:36.577972 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a6906-3977-46e2-a72b-68f03899f495 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:36.582706 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab78ebb-f86b-447b-be89-a019b7b6d453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:36.583378 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab78ebb-f86b-447b-be89-a019b7b6d453 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:36.583685 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab78ebb-f86b-447b-be89-a019b7b6d453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:36.587505 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab78ebb-f86b-447b-be89-a019b7b6d453 admin admin] PATH_INFO: `/v3/projects/60521ccbb32e481e95a33085b4a52adc/users/164b2c2fa1e841f7a0b8896afbde4bb3/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:36.587505 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab78ebb-f86b-447b-be89-a019b7b6d453 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60521ccbb32e481e95a33085b4a52adc, user_id=164b2c2fa1e841f7a0b8896afbde4bb3, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:36.589534 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab78ebb-f86b-447b-be89-a019b7b6d453 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:36.600524 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1378/2741] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:36 2022] PUT /identity/v3/projects/60521ccbb32e481e95a33085b4a52adc/users/164b2c2fa1e841f7a0b8896afbde4bb3/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:36.611445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817ee0b-a924-4456-9269-64b45938182d None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:36.619214 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817ee0b-a924-4456-9269-64b45938182d None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:36.619214 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817ee0b-a924-4456-9269-64b45938182d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:36.639523 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-3817ee0b-a924-4456-9269-64b45938182d None None] MFA Rules not processed for user `164b2c2fa1e841f7a0b8896afbde4bb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:36.640681 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-3817ee0b-a924-4456-9269-64b45938182d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:36.710209 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1364/2742] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:36 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:36.722942 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab78ebb-f86b-447b-be89-a019b7b6d453 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:36.782042 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab78ebb-f86b-447b-be89-a019b7b6d453 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:36.785751 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239d079-d9df-4469-abab-bc2557c25d04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:36.786403 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239d079-d9df-4469-abab-bc2557c25d04 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:36.786683 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239d079-d9df-4469-abab-bc2557c25d04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:36.786955 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a239d079-d9df-4469-abab-bc2557c25d04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:36.796705 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a239d079-d9df-4469-abab-bc2557c25d04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:36.798162 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a239d079-d9df-4469-abab-bc2557c25d04 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:36.807867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1379/2743] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:36 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:19:36.975950 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3817ee0b-a924-4456-9269-64b45938182d None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:36.986119 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3817ee0b-a924-4456-9269-64b45938182d None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:36.989708 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5e5a3-3eab-4edf-9ab3-6fdbd4e031e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:36.990361 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5e5a3-3eab-4edf-9ab3-6fdbd4e031e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:36.990643 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5e5a3-3eab-4edf-9ab3-6fdbd4e031e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:36.990918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5e5a3-3eab-4edf-9ab3-6fdbd4e031e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:37.048248 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5e5a3-3eab-4edf-9ab3-6fdbd4e031e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:37.049732 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5e5a3-3eab-4edf-9ab3-6fdbd4e031e4 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:37.059278 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1365/2744] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:19:37.568339 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239d079-d9df-4469-abab-bc2557c25d04 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:37.569721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-a239d079-d9df-4469-abab-bc2557c25d04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:37.580996 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a239d079-d9df-4469-abab-bc2557c25d04 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:37.584583 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f634f7bc-95c4-4225-940e-1f0c066acd3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:37.585186 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634f7bc-95c4-4225-940e-1f0c066acd3b service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:37.585465 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634f7bc-95c4-4225-940e-1f0c066acd3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:37.585737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f634f7bc-95c4-4225-940e-1f0c066acd3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:37.587477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f634f7bc-95c4-4225-940e-1f0c066acd3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:37.611327 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:37.616563 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5e5a3-3eab-4edf-9ab3-6fdbd4e031e4 service cinder] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:37.644455 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:37.649508 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f634f7bc-95c4-4225-940e-1f0c066acd3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:37.650990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f634f7bc-95c4-4225-940e-1f0c066acd3b service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:37.660695 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1380/2745] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:19:37.682677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5e5a3-3eab-4edf-9ab3-6fdbd4e031e4 service cinder] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:37.687528 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7a73f-f7ea-4473-8673-05dfc186f2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:37.688203 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7a73f-f7ea-4473-8673-05dfc186f2b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:37.688489 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7a73f-f7ea-4473-8673-05dfc186f2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:37.688772 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7a73f-f7ea-4473-8673-05dfc186f2b4 admin admin] PATH_INFO: `/v3/users/6bbde9f16c0e45b7b3515380e608d9af` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:37.689982 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7a73f-f7ea-4473-8673-05dfc186f2b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bbde9f16c0e45b7b3515380e608d9af)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:37.692096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7a73f-f7ea-4473-8673-05dfc186f2b4 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:37.943255 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-c0f7a73f-f7ea-4473-8673-05dfc186f2b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bbde9f16c0e45b7b3515380e608d9af'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:37.950368 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-c0f7a73f-f7ea-4473-8673-05dfc186f2b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bbde9f16c0e45b7b3515380e608d9af'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:37.951053 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-c0f7a73f-f7ea-4473-8673-05dfc186f2b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bbde9f16c0e45b7b3515380e608d9af'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:38.014017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-c0f7a73f-f7ea-4473-8673-05dfc186f2b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bbde9f16c0e45b7b3515380e608d9af'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:38.027276 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-c0f7a73f-f7ea-4473-8673-05dfc186f2b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bbde9f16c0e45b7b3515380e608d9af'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:38.033267 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1366/2746] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:19:37 2022] DELETE /identity/v3/users/6bbde9f16c0e45b7b3515380e608d9af => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:19:38.037863 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fee8d-733b-4358-add0-43b58456a302 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:38.038184 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fee8d-733b-4358-add0-43b58456a302 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:38.038462 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2fee8d-733b-4358-add0-43b58456a302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:38.058145 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-ee2fee8d-733b-4358-add0-43b58456a302 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:38.059336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ee2fee8d-733b-4358-add0-43b58456a302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:38.118105 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1381/2747] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:38.136950 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7a73f-f7ea-4473-8673-05dfc186f2b4 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:38.138514 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-c0f7a73f-f7ea-4473-8673-05dfc186f2b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:38.196900 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7a73f-f7ea-4473-8673-05dfc186f2b4 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:38.200534 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61a4b5-e623-43b1-b7b2-738b2d56aa38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:38.201187 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61a4b5-e623-43b1-b7b2-738b2d56aa38 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:38.201468 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61a4b5-e623-43b1-b7b2-738b2d56aa38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:38.201743 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61a4b5-e623-43b1-b7b2-738b2d56aa38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:38.210868 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61a4b5-e623-43b1-b7b2-738b2d56aa38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:38.212302 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61a4b5-e623-43b1-b7b2-738b2d56aa38 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:38.222430 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1367/2748] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:19:38.649915 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2fee8d-733b-4358-add0-43b58456a302 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:38.652051 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ee2fee8d-733b-4358-add0-43b58456a302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:38.665380 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2fee8d-733b-4358-add0-43b58456a302 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:38.670424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc608f4e-bb7a-4c4e-982b-e28b9401cde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:38.671005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc608f4e-bb7a-4c4e-982b-e28b9401cde5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:38.671288 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc608f4e-bb7a-4c4e-982b-e28b9401cde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:38.671659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc608f4e-bb7a-4c4e-982b-e28b9401cde5 admin admin] PATH_INFO: `/v3/projects/4686c7a4d6d844bfa8bb94ac7f814d55` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:38.672821 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc608f4e-bb7a-4c4e-982b-e28b9401cde5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4686c7a4d6d844bfa8bb94ac7f814d55)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:38.675396 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc608f4e-bb7a-4c4e-982b-e28b9401cde5 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:38.787487 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-bc608f4e-bb7a-4c4e-982b-e28b9401cde5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4686c7a4d6d844bfa8bb94ac7f814d55', 'request_id': 'req-bc608f4e-bb7a-4c4e-982b-e28b9401cde5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:38.793678 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-bc608f4e-bb7a-4c4e-982b-e28b9401cde5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4686c7a4d6d844bfa8bb94ac7f814d55', 'request_id': 'req-bc608f4e-bb7a-4c4e-982b-e28b9401cde5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:38.903838 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-bc608f4e-bb7a-4c4e-982b-e28b9401cde5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4686c7a4d6d844bfa8bb94ac7f814d55', 'request_id': 'req-bc608f4e-bb7a-4c4e-982b-e28b9401cde5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:38.915116 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-bc608f4e-bb7a-4c4e-982b-e28b9401cde5 admin admin] The token cache is being invalidate because project 4686c7a4d6d844bfa8bb94ac7f814d55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:19:38.915409 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-bc608f4e-bb7a-4c4e-982b-e28b9401cde5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:38.916152 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-bc608f4e-bb7a-4c4e-982b-e28b9401cde5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:38.917733 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1382/2749] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:19:38 2022] DELETE /identity/v3/projects/4686c7a4d6d844bfa8bb94ac7f814d55 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:38.921852 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61a4b5-e623-43b1-b7b2-738b2d56aa38 service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:38.933163 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61a4b5-e623-43b1-b7b2-738b2d56aa38 service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:38.983586 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:38.996743 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023a765-94b4-40be-b729-665eb5ba2c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:38.997461 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023a765-94b4-40be-b729-665eb5ba2c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:38.997811 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023a765-94b4-40be-b729-665eb5ba2c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:38.998182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9023a765-94b4-40be-b729-665eb5ba2c78 admin admin] PATH_INFO: `/v3/users/164b2c2fa1e841f7a0b8896afbde4bb3` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:38.999335 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023a765-94b4-40be-b729-665eb5ba2c78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=164b2c2fa1e841f7a0b8896afbde4bb3)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:39.001294 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9023a765-94b4-40be-b729-665eb5ba2c78 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:39.139452 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc608f4e-bb7a-4c4e-982b-e28b9401cde5 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:39.140837 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bc608f4e-bb7a-4c4e-982b-e28b9401cde5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:39.194184 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:39.199312 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc608f4e-bb7a-4c4e-982b-e28b9401cde5 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:39.203072 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56aac15-fb5c-40a4-831c-9f4fbb4bb1a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:39.203694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56aac15-fb5c-40a4-831c-9f4fbb4bb1a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:39.203973 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56aac15-fb5c-40a4-831c-9f4fbb4bb1a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:39.204244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56aac15-fb5c-40a4-831c-9f4fbb4bb1a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:39.206180 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e56aac15-fb5c-40a4-831c-9f4fbb4bb1a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:39.255670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9023a765-94b4-40be-b729-665eb5ba2c78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '164b2c2fa1e841f7a0b8896afbde4bb3'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:39.263077 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9023a765-94b4-40be-b729-665eb5ba2c78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '164b2c2fa1e841f7a0b8896afbde4bb3'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:39.263750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9023a765-94b4-40be-b729-665eb5ba2c78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '164b2c2fa1e841f7a0b8896afbde4bb3'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:39.269495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56aac15-fb5c-40a4-831c-9f4fbb4bb1a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:39.270998 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56aac15-fb5c-40a4-831c-9f4fbb4bb1a3 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:39.280753 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1383/2750] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:19:39.332647 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9023a765-94b4-40be-b729-665eb5ba2c78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '164b2c2fa1e841f7a0b8896afbde4bb3'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:39.345555 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9023a765-94b4-40be-b729-665eb5ba2c78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '164b2c2fa1e841f7a0b8896afbde4bb3'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:39.351675 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1368/2751] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:19:38 2022] DELETE /identity/v3/users/164b2c2fa1e841f7a0b8896afbde4bb3 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:39.793360 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56aac15-fb5c-40a4-831c-9f4fbb4bb1a3 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:39.794807 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-e56aac15-fb5c-40a4-831c-9f4fbb4bb1a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:39.809278 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56aac15-fb5c-40a4-831c-9f4fbb4bb1a3 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:39.870158 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367cea4-2834-4d8a-a057-297aec8d2278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:39.870893 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367cea4-2834-4d8a-a057-297aec8d2278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:39.871200 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367cea4-2834-4d8a-a057-297aec8d2278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:39.871477 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367cea4-2834-4d8a-a057-297aec8d2278 admin admin] PATH_INFO: `/v3/projects/60521ccbb32e481e95a33085b4a52adc` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:39.872631 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367cea4-2834-4d8a-a057-297aec8d2278 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60521ccbb32e481e95a33085b4a52adc)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:39.874650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367cea4-2834-4d8a-a057-297aec8d2278 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:39.974174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4367cea4-2834-4d8a-a057-297aec8d2278 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60521ccbb32e481e95a33085b4a52adc', 'request_id': 'req-4367cea4-2834-4d8a-a057-297aec8d2278'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:39.980504 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4367cea4-2834-4d8a-a057-297aec8d2278 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60521ccbb32e481e95a33085b4a52adc', 'request_id': 'req-4367cea4-2834-4d8a-a057-297aec8d2278'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:40.059191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4367cea4-2834-4d8a-a057-297aec8d2278 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60521ccbb32e481e95a33085b4a52adc', 'request_id': 'req-4367cea4-2834-4d8a-a057-297aec8d2278'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:40.070093 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4367cea4-2834-4d8a-a057-297aec8d2278 admin admin] The token cache is being invalidate because project 60521ccbb32e481e95a33085b4a52adc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80328) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:19:40.070386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4367cea4-2834-4d8a-a057-297aec8d2278 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:40.071021 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4367cea4-2834-4d8a-a057-297aec8d2278 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:40.072569 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1384/2752] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:19:39 2022] DELETE /identity/v3/projects/60521ccbb32e481e95a33085b4a52adc => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:40.508114 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023a765-94b4-40be-b729-665eb5ba2c78 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:40.509522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-9023a765-94b4-40be-b729-665eb5ba2c78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:40.571138 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9023a765-94b4-40be-b729-665eb5ba2c78 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:40.622113 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c1fd3-994c-4a27-aaad-5c4d4b30698d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:40.622692 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c1fd3-994c-4a27-aaad-5c4d4b30698d service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:40.622981 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c1fd3-994c-4a27-aaad-5c4d4b30698d service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:40.623258 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c1fd3-994c-4a27-aaad-5c4d4b30698d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:40.625009 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-d30c1fd3-994c-4a27-aaad-5c4d4b30698d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:40.660482 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:40.686689 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30c1fd3-994c-4a27-aaad-5c4d4b30698d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:40.688132 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30c1fd3-994c-4a27-aaad-5c4d4b30698d service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:40.697256 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1369/2753] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:40 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:19:48.138203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367cea4-2834-4d8a-a057-297aec8d2278 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:48.139668 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4367cea4-2834-4d8a-a057-297aec8d2278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:48.169967 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:48.206540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367cea4-2834-4d8a-a057-297aec8d2278 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:48.212053 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8f619-9de4-43f1-8282-899c3dae177e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:48.212694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8f619-9de4-43f1-8282-899c3dae177e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:48.212976 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8f619-9de4-43f1-8282-899c3dae177e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:48.213856 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8f619-9de4-43f1-8282-899c3dae177e admin admin] PATH_INFO: `/v3/users/e49214d9458b4715a6c05a5cf7fdb190` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:48.215173 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8f619-9de4-43f1-8282-899c3dae177e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e49214d9458b4715a6c05a5cf7fdb190)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:48.217097 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8f619-9de4-43f1-8282-899c3dae177e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:48.416075 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1fc8f619-9de4-43f1-8282-899c3dae177e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e49214d9458b4715a6c05a5cf7fdb190'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:48.422916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1fc8f619-9de4-43f1-8282-899c3dae177e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e49214d9458b4715a6c05a5cf7fdb190'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:48.423596 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1fc8f619-9de4-43f1-8282-899c3dae177e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e49214d9458b4715a6c05a5cf7fdb190'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:48.436391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1fc8f619-9de4-43f1-8282-899c3dae177e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e49214d9458b4715a6c05a5cf7fdb190'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:48.437053 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1fc8f619-9de4-43f1-8282-899c3dae177e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e49214d9458b4715a6c05a5cf7fdb190'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:48.509398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1385/2754] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:19:48 2022] DELETE /identity/v3/users/e49214d9458b4715a6c05a5cf7fdb190 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:48.514619 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2afb36-898c-456c-965c-7a8441ed2230 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:48.515136 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2afb36-898c-456c-965c-7a8441ed2230 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:48.515237 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2afb36-898c-456c-965c-7a8441ed2230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:48.534807 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-3e2afb36-898c-456c-965c-7a8441ed2230 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:48.536016 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-3e2afb36-898c-456c-965c-7a8441ed2230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:48.596107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1370/2755] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:19:48.605695 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8f619-9de4-43f1-8282-899c3dae177e admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:48.607135 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1fc8f619-9de4-43f1-8282-899c3dae177e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:48.665857 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8f619-9de4-43f1-8282-899c3dae177e admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:48.669588 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e98f16-7127-4149-b217-989f70943c0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:48.670174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e98f16-7127-4149-b217-989f70943c0d service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:48.670457 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e98f16-7127-4149-b217-989f70943c0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:48.670730 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e98f16-7127-4149-b217-989f70943c0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:48.679731 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e98f16-7127-4149-b217-989f70943c0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:48.681541 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e98f16-7127-4149-b217-989f70943c0d service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:48.692006 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1386/2756] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:19:49.221091 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2afb36-898c-456c-965c-7a8441ed2230 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:49.222829 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-3e2afb36-898c-456c-965c-7a8441ed2230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:49.236008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2afb36-898c-456c-965c-7a8441ed2230 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:49.240824 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041b6a24-f687-48a3-9119-36f6597fda77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:49.241612 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b6a24-f687-48a3-9119-36f6597fda77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:49.241962 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b6a24-f687-48a3-9119-36f6597fda77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:49.242244 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b6a24-f687-48a3-9119-36f6597fda77 admin admin] PATH_INFO: `/v3/projects/012a8fb9ecc940edb934c78533d603c2` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:49.243485 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041b6a24-f687-48a3-9119-36f6597fda77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=012a8fb9ecc940edb934c78533d603c2)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:49.246098 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041b6a24-f687-48a3-9119-36f6597fda77 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:49.358158 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-041b6a24-f687-48a3-9119-36f6597fda77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '012a8fb9ecc940edb934c78533d603c2', 'request_id': 'req-041b6a24-f687-48a3-9119-36f6597fda77'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:49.364466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-041b6a24-f687-48a3-9119-36f6597fda77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '012a8fb9ecc940edb934c78533d603c2', 'request_id': 'req-041b6a24-f687-48a3-9119-36f6597fda77'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:49.370205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-041b6a24-f687-48a3-9119-36f6597fda77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '012a8fb9ecc940edb934c78533d603c2', 'request_id': 'req-041b6a24-f687-48a3-9119-36f6597fda77'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:49.447902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-041b6a24-f687-48a3-9119-36f6597fda77 admin admin] The token cache is being invalidate because project 012a8fb9ecc940edb934c78533d603c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:19:49.448196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-041b6a24-f687-48a3-9119-36f6597fda77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:49.448898 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-041b6a24-f687-48a3-9119-36f6597fda77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:49.450817 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1371/2757] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:19:49 2022] DELETE /identity/v3/projects/012a8fb9ecc940edb934c78533d603c2 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:49.454535 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e98f16-7127-4149-b217-989f70943c0d service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:49.464972 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e98f16-7127-4149-b217-989f70943c0d service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:49.510014 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:49.521784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37b020-bf1a-4176-9c30-4b7292856133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:49.522466 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37b020-bf1a-4176-9c30-4b7292856133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:49.522751 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37b020-bf1a-4176-9c30-4b7292856133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:49.523025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37b020-bf1a-4176-9c30-4b7292856133 admin admin] PATH_INFO: `/v3/users/4c620a476663412791ac70732d1e1741` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:49.524188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37b020-bf1a-4176-9c30-4b7292856133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c620a476663412791ac70732d1e1741)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:49.526680 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37b020-bf1a-4176-9c30-4b7292856133 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:49.661979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9c37b020-bf1a-4176-9c30-4b7292856133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c620a476663412791ac70732d1e1741'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:49.668886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9c37b020-bf1a-4176-9c30-4b7292856133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c620a476663412791ac70732d1e1741'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:49.669558 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9c37b020-bf1a-4176-9c30-4b7292856133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c620a476663412791ac70732d1e1741'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:49.681868 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9c37b020-bf1a-4176-9c30-4b7292856133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c620a476663412791ac70732d1e1741'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:49.682528 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9c37b020-bf1a-4176-9c30-4b7292856133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c620a476663412791ac70732d1e1741'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:49.750685 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1387/2758] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:19:49 2022] DELETE /identity/v3/users/4c620a476663412791ac70732d1e1741 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:19:50.183107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041b6a24-f687-48a3-9119-36f6597fda77 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:50.184849 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-041b6a24-f687-48a3-9119-36f6597fda77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:50.205236 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041b6a24-f687-48a3-9119-36f6597fda77 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:50.266861 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268875b-8370-47d9-86f0-bcab74e62cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:50.267412 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268875b-8370-47d9-86f0-bcab74e62cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:50.267694 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268875b-8370-47d9-86f0-bcab74e62cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:50.267974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268875b-8370-47d9-86f0-bcab74e62cb0 admin admin] PATH_INFO: `/v3/projects/2c0ac06be2734275872d94bb2760a1e0` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:50.269129 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268875b-8370-47d9-86f0-bcab74e62cb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c0ac06be2734275872d94bb2760a1e0)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:50.271065 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9268875b-8370-47d9-86f0-bcab74e62cb0 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:50.272501 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:50.373345 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9268875b-8370-47d9-86f0-bcab74e62cb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c0ac06be2734275872d94bb2760a1e0', 'request_id': 'req-9268875b-8370-47d9-86f0-bcab74e62cb0'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:50.379398 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9268875b-8370-47d9-86f0-bcab74e62cb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c0ac06be2734275872d94bb2760a1e0', 'request_id': 'req-9268875b-8370-47d9-86f0-bcab74e62cb0'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:50.385645 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9268875b-8370-47d9-86f0-bcab74e62cb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c0ac06be2734275872d94bb2760a1e0', 'request_id': 'req-9268875b-8370-47d9-86f0-bcab74e62cb0'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:50.480116 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9268875b-8370-47d9-86f0-bcab74e62cb0 admin admin] The token cache is being invalidate because project 2c0ac06be2734275872d94bb2760a1e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:19:50.480437 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9268875b-8370-47d9-86f0-bcab74e62cb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:50.481068 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-9268875b-8370-47d9-86f0-bcab74e62cb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:50.482654 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1372/2759] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:19:50 2022] DELETE /identity/v3/projects/2c0ac06be2734275872d94bb2760a1e0 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:50.486983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37b020-bf1a-4176-9c30-4b7292856133 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:50.498058 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9c37b020-bf1a-4176-9c30-4b7292856133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:50.501460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37b020-bf1a-4176-9c30-4b7292856133 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:50.557579 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8564503f-50af-4dd4-803b-f1d337e74489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:50.558179 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564503f-50af-4dd4-803b-f1d337e74489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:50.558473 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564503f-50af-4dd4-803b-f1d337e74489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:50.558736 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564503f-50af-4dd4-803b-f1d337e74489 admin admin] PATH_INFO: `/v3/users/23f55fbd296c4851aa27fdf78e039588` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:50.559905 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8564503f-50af-4dd4-803b-f1d337e74489 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23f55fbd296c4851aa27fdf78e039588)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:50.561819 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8564503f-50af-4dd4-803b-f1d337e74489 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:50.704289 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8564503f-50af-4dd4-803b-f1d337e74489 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23f55fbd296c4851aa27fdf78e039588'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:50.711185 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8564503f-50af-4dd4-803b-f1d337e74489 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23f55fbd296c4851aa27fdf78e039588'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:50.712000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8564503f-50af-4dd4-803b-f1d337e74489 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23f55fbd296c4851aa27fdf78e039588'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:50.723745 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:50.726038 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8564503f-50af-4dd4-803b-f1d337e74489 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23f55fbd296c4851aa27fdf78e039588'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:50.726749 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-8564503f-50af-4dd4-803b-f1d337e74489 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23f55fbd296c4851aa27fdf78e039588'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:50.790510 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1388/2760] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:19:50 2022] DELETE /identity/v3/users/23f55fbd296c4851aa27fdf78e039588 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:51.191310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268875b-8370-47d9-86f0-bcab74e62cb0 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:51.192787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-9268875b-8370-47d9-86f0-bcab74e62cb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:51.206325 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268875b-8370-47d9-86f0-bcab74e62cb0 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:51.264842 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df4805-a38a-4573-9d46-0a301b86f3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:51.265491 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df4805-a38a-4573-9d46-0a301b86f3b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:51.265776 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df4805-a38a-4573-9d46-0a301b86f3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:51.266095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df4805-a38a-4573-9d46-0a301b86f3b6 admin admin] PATH_INFO: `/v3/projects/fd0c006a2d534dbf82fea5a4d86f571b` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:51.267250 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df4805-a38a-4573-9d46-0a301b86f3b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd0c006a2d534dbf82fea5a4d86f571b)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:51.269151 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df4805-a38a-4573-9d46-0a301b86f3b6 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:51.393570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-07df4805-a38a-4573-9d46-0a301b86f3b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd0c006a2d534dbf82fea5a4d86f571b', 'request_id': 'req-07df4805-a38a-4573-9d46-0a301b86f3b6'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:51.399977 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-07df4805-a38a-4573-9d46-0a301b86f3b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd0c006a2d534dbf82fea5a4d86f571b', 'request_id': 'req-07df4805-a38a-4573-9d46-0a301b86f3b6'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:51.406268 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-07df4805-a38a-4573-9d46-0a301b86f3b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd0c006a2d534dbf82fea5a4d86f571b', 'request_id': 'req-07df4805-a38a-4573-9d46-0a301b86f3b6'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:51.465652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-07df4805-a38a-4573-9d46-0a301b86f3b6 admin admin] The token cache is being invalidate because project fd0c006a2d534dbf82fea5a4d86f571b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:19:51.465979 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-07df4805-a38a-4573-9d46-0a301b86f3b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:51.466622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-07df4805-a38a-4573-9d46-0a301b86f3b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:19:51.468201 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1373/2761] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:19:51 2022] DELETE /identity/v3/projects/fd0c006a2d534dbf82fea5a4d86f571b => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:51.473390 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4591c-b320-4d86-9150-a480e1d14ae2 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:51.473672 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4591c-b320-4d86-9150-a480e1d14ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:51.473983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f4591c-b320-4d86-9150-a480e1d14ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:51.493563 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-05f4591c-b320-4d86-9150-a480e1d14ae2 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:51.494967 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-05f4591c-b320-4d86-9150-a480e1d14ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:51.554768 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1389/2762] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:51.563465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40a528-f6de-4165-a797-a6c7f56a06ce None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:51.563759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40a528-f6de-4165-a797-a6c7f56a06ce None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:51.564076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f40a528-f6de-4165-a797-a6c7f56a06ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:51.585396 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-2f40a528-f6de-4165-a797-a6c7f56a06ce None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:51.591648 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-2f40a528-f6de-4165-a797-a6c7f56a06ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:51.594478 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1374/2763] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:51.599013 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4591c-b320-4d86-9150-a480e1d14ae2 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:51.609036 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f4591c-b320-4d86-9150-a480e1d14ae2 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:51.613733 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f7e54-a8c5-45fb-9b02-8808c330b6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:51.614386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f7e54-a8c5-45fb-9b02-8808c330b6d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:51.614674 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f7e54-a8c5-45fb-9b02-8808c330b6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:51.614948 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f7e54-a8c5-45fb-9b02-8808c330b6d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:51.615648 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f7e54-a8c5-45fb-9b02-8808c330b6d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:19:51.615958 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f7e54-a8c5-45fb-9b02-8808c330b6d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:51.617559 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f7e54-a8c5-45fb-9b02-8808c330b6d1 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:51.630651 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1390/2764] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:19:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:19:51.635825 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40a528-f6de-4165-a797-a6c7f56a06ce None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:51.649455 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f40a528-f6de-4165-a797-a6c7f56a06ce None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:51.650431 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2a504-ad1b-46a2-ae67-2a25675ac81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:51.651166 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2a504-ad1b-46a2-ae67-2a25675ac81f admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:51.651452 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2a504-ad1b-46a2-ae67-2a25675ac81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:51.651736 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f2a504-ad1b-46a2-ae67-2a25675ac81f admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:51.652442 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f2a504-ad1b-46a2-ae67-2a25675ac81f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:51.654431 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f2a504-ad1b-46a2-ae67-2a25675ac81f admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:51.666193 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:51.781415 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1375/2765] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:19:51 2022] POST /identity/v3/projects => generated 391 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:19:51.786038 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f7e54-a8c5-45fb-9b02-8808c330b6d1 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:51.796746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f7e54-a8c5-45fb-9b02-8808c330b6d1 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:51.852945 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463bb0e-7f22-4fbf-b815-c13d4941aae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:51.853587 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463bb0e-7f22-4fbf-b815-c13d4941aae5 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:51.853874 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463bb0e-7f22-4fbf-b815-c13d4941aae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:51.854182 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463bb0e-7f22-4fbf-b815-c13d4941aae5 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:51.854854 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d463bb0e-7f22-4fbf-b815-c13d4941aae5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:51.856807 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d463bb0e-7f22-4fbf-b815-c13d4941aae5 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:51.924004 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1391/2766] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:19:51 2022] POST /identity/v3/users => generated 296 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:19:51.928734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2a504-ad1b-46a2-ae67-2a25675ac81f admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:51.941212 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f2a504-ad1b-46a2-ae67-2a25675ac81f admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:51.945992 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b432361c-46ef-49ab-a79a-c92adab2b0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:51.946647 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432361c-46ef-49ab-a79a-c92adab2b0b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:51.946954 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432361c-46ef-49ab-a79a-c92adab2b0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:51.947249 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432361c-46ef-49ab-a79a-c92adab2b0b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:51.947947 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b432361c-46ef-49ab-a79a-c92adab2b0b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:51.949549 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b432361c-46ef-49ab-a79a-c92adab2b0b9 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:51.961093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1376/2767] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:19:51.965128 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463bb0e-7f22-4fbf-b815-c13d4941aae5 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:51.975430 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463bb0e-7f22-4fbf-b815-c13d4941aae5 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:51.980780 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88acc47b-ab76-4c12-aa1b-28ffeaae0051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:51.986795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acc47b-ab76-4c12-aa1b-28ffeaae0051 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:51.986795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acc47b-ab76-4c12-aa1b-28ffeaae0051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:51.986795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acc47b-ab76-4c12-aa1b-28ffeaae0051 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:51.986795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88acc47b-ab76-4c12-aa1b-28ffeaae0051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:51.986795 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88acc47b-ab76-4c12-aa1b-28ffeaae0051 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:51.995712 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1392/2768] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:19:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:19:52.000221 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b432361c-46ef-49ab-a79a-c92adab2b0b9 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:52.011025 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b432361c-46ef-49ab-a79a-c92adab2b0b9 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:52.015821 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799ac61-597a-4ab7-9e58-6a2bf6daa6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:52.016574 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799ac61-597a-4ab7-9e58-6a2bf6daa6da admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:52.016851 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799ac61-597a-4ab7-9e58-6a2bf6daa6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:52.017128 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799ac61-597a-4ab7-9e58-6a2bf6daa6da admin admin] PATH_INFO: `/v3/projects/c736382d431f43e6910e7f69601e2c3e/users/a5bdb60473494e748144e636d339d580/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:52.033017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9799ac61-597a-4ab7-9e58-6a2bf6daa6da admin admin] RBAC: Authorizing `identity:create_grant(project_id=c736382d431f43e6910e7f69601e2c3e, user_id=a5bdb60473494e748144e636d339d580, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:52.035994 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9799ac61-597a-4ab7-9e58-6a2bf6daa6da admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:52.127849 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1377/2769] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:19:51 2022] PUT /identity/v3/projects/c736382d431f43e6910e7f69601e2c3e/users/a5bdb60473494e748144e636d339d580/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:19:52.138475 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced33560-a804-489f-aeff-af9f3c656f60 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:52.138766 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced33560-a804-489f-aeff-af9f3c656f60 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:52.139044 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced33560-a804-489f-aeff-af9f3c656f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:52.170315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-ced33560-a804-489f-aeff-af9f3c656f60 None None] MFA Rules not processed for user `a5bdb60473494e748144e636d339d580`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:52.171600 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ced33560-a804-489f-aeff-af9f3c656f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:52.215991 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:19:52.246499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1393/2770] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:52 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:19:52.259350 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799ac61-597a-4ab7-9e58-6a2bf6daa6da admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:52.262538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-9799ac61-597a-4ab7-9e58-6a2bf6daa6da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:52.301481 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76fe23-dce0-4909-ad19-62deaba911ac None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:52.302125 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76fe23-dce0-4909-ad19-62deaba911ac None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:52.302572 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76fe23-dce0-4909-ad19-62deaba911ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:52.325785 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799ac61-597a-4ab7-9e58-6a2bf6daa6da admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:52.330279 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-bd76fe23-dce0-4909-ad19-62deaba911ac None None] MFA Rules not processed for user `02569c995a6d4b678933599f60ad3909`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:19:52.330460 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cb71f-a992-4c28-a81a-36ae630245ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:52.331050 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cb71f-a992-4c28-a81a-36ae630245ae service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:52.331379 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cb71f-a992-4c28-a81a-36ae630245ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:52.331574 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bd76fe23-dce0-4909-ad19-62deaba911ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:52.331715 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cb71f-a992-4c28-a81a-36ae630245ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:52.341412 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16cb71f-a992-4c28-a81a-36ae630245ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:52.343556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16cb71f-a992-4c28-a81a-36ae630245ae service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:52.353169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1378/2771] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:52 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:19:52.393873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1394/2772] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:19:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:19:52.733403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cb71f-a992-4c28-a81a-36ae630245ae service cinder] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:19:52.734713 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-b16cb71f-a992-4c28-a81a-36ae630245ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:52.816980 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cb71f-a992-4c28-a81a-36ae630245ae service cinder] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:19:52.820691 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe774e-d63b-487b-a7e2-19b829eb9997 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:19:52.821350 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe774e-d63b-487b-a7e2-19b829eb9997 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:19:52.821631 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe774e-d63b-487b-a7e2-19b829eb9997 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:19:52.821906 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe774e-d63b-487b-a7e2-19b829eb9997 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:19:52.824503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-edfe774e-d63b-487b-a7e2-19b829eb9997 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:19:52.835509 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe774e-d63b-487b-a7e2-19b829eb9997 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:19:52.837024 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe774e-d63b-487b-a7e2-19b829eb9997 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:19:52.846753 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1379/2773] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:19:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:20:02.386110 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=33 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:20:02.842533 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=70,keystone:INSERT=2 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:20:08.416370 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76fe23-dce0-4909-ad19-62deaba911ac None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:08.426187 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd76fe23-dce0-4909-ad19-62deaba911ac None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:08.429941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe763fa-82c0-4009-b10a-dcccec002bf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:08.430419 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe763fa-82c0-4009-b10a-dcccec002bf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:08.430983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe763fa-82c0-4009-b10a-dcccec002bf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:08.431254 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe763fa-82c0-4009-b10a-dcccec002bf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:08.433020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2fe763fa-82c0-4009-b10a-dcccec002bf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:08.491648 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe763fa-82c0-4009-b10a-dcccec002bf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:08.493073 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe763fa-82c0-4009-b10a-dcccec002bf4 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:08.501933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1395/2774] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:20:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:20:09.382952 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe774e-d63b-487b-a7e2-19b829eb9997 service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:09.384767 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-edfe774e-d63b-487b-a7e2-19b829eb9997 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:09.467276 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe774e-d63b-487b-a7e2-19b829eb9997 service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:09.473133 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3992e01-1501-49dc-9a53-d6729adfe2c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:09.473760 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3992e01-1501-49dc-9a53-d6729adfe2c8 service nova] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:09.474073 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3992e01-1501-49dc-9a53-d6729adfe2c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:09.474344 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3992e01-1501-49dc-9a53-d6729adfe2c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:09.476115 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e3992e01-1501-49dc-9a53-d6729adfe2c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:09.489182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3992e01-1501-49dc-9a53-d6729adfe2c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:09.490751 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3992e01-1501-49dc-9a53-d6729adfe2c8 service nova] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:09.502463 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1380/2775] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:20:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:20:14.877871 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe763fa-82c0-4009-b10a-dcccec002bf4 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:14.879315 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-2fe763fa-82c0-4009-b10a-dcccec002bf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:14.949379 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe763fa-82c0-4009-b10a-dcccec002bf4 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:14.954671 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8bb3e2-59ed-4efb-af5b-7256bb25ac1a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad5cc9927374836b0cc74db85b686fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:14.955322 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bb3e2-59ed-4efb-af5b-7256bb25ac1a service placement] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:14.955606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bb3e2-59ed-4efb-af5b-7256bb25ac1a service placement] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:14.955882 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8bb3e2-59ed-4efb-af5b-7256bb25ac1a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:14.957672 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-ae8bb3e2-59ed-4efb-af5b-7256bb25ac1a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:14.969916 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8bb3e2-59ed-4efb-af5b-7256bb25ac1a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:14.971554 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8bb3e2-59ed-4efb-af5b-7256bb25ac1a service placement] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:14.983102 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1396/2776] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:20:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:20:16.213360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3992e01-1501-49dc-9a53-d6729adfe2c8 service nova] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:16.214864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e3992e01-1501-49dc-9a53-d6729adfe2c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:16.226856 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3992e01-1501-49dc-9a53-d6729adfe2c8 service nova] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:16.231053 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8f5e4-e697-4639-b0f9-0a73c6fbfb80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:16.231663 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8f5e4-e697-4639-b0f9-0a73c6fbfb80 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:16.231967 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8f5e4-e697-4639-b0f9-0a73c6fbfb80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:16.232229 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8f5e4-e697-4639-b0f9-0a73c6fbfb80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:16.234005 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-cfc8f5e4-e697-4639-b0f9-0a73c6fbfb80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:16.238981 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8bb3e2-59ed-4efb-af5b-7256bb25ac1a service placement] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:16.301345 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8f5e4-e697-4639-b0f9-0a73c6fbfb80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:16.302813 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8f5e4-e697-4639-b0f9-0a73c6fbfb80 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:16.303158 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8bb3e2-59ed-4efb-af5b-7256bb25ac1a service placement] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:16.308926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ef79a-373a-48e6-95fd-8c0981aeef1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:16.309564 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ef79a-373a-48e6-95fd-8c0981aeef1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:16.309839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ef79a-373a-48e6-95fd-8c0981aeef1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:16.310307 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9ef79a-373a-48e6-95fd-8c0981aeef1e admin admin] PATH_INFO: `/v3/users/a5bdb60473494e748144e636d339d580` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:16.311430 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ef79a-373a-48e6-95fd-8c0981aeef1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5bdb60473494e748144e636d339d580)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:16.313317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9ef79a-373a-48e6-95fd-8c0981aeef1e admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:16.313861 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1381/2777] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:20:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:20:16.515432 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:20:16.528606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-da9ef79a-373a-48e6-95fd-8c0981aeef1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5bdb60473494e748144e636d339d580'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:16.535683 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-da9ef79a-373a-48e6-95fd-8c0981aeef1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5bdb60473494e748144e636d339d580'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:16.536473 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-da9ef79a-373a-48e6-95fd-8c0981aeef1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5bdb60473494e748144e636d339d580'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:16.548980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-da9ef79a-373a-48e6-95fd-8c0981aeef1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5bdb60473494e748144e636d339d580'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:16.549672 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-da9ef79a-373a-48e6-95fd-8c0981aeef1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5bdb60473494e748144e636d339d580'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:16.624543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1397/2778] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:20:16 2022] DELETE /identity/v3/users/a5bdb60473494e748144e636d339d580 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:20:16.629261 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f669b6-2632-47b8-b0d5-807522427842 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:16.629554 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f669b6-2632-47b8-b0d5-807522427842 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:16.629939 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f669b6-2632-47b8-b0d5-807522427842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:16.650862 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-72f669b6-2632-47b8-b0d5-807522427842 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:20:16.652122 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-72f669b6-2632-47b8-b0d5-807522427842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:16.734389 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1382/2779] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:20:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:20:16.746665 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ef79a-373a-48e6-95fd-8c0981aeef1e admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:16.748152 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-da9ef79a-373a-48e6-95fd-8c0981aeef1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:16.828013 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9ef79a-373a-48e6-95fd-8c0981aeef1e admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:16.832717 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce0800e-974c-4164-9b8a-a0be7721f99a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:16.833474 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0800e-974c-4164-9b8a-a0be7721f99a service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:16.833803 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0800e-974c-4164-9b8a-a0be7721f99a service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:16.834121 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce0800e-974c-4164-9b8a-a0be7721f99a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:16.844157 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0800e-974c-4164-9b8a-a0be7721f99a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:16.845580 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce0800e-974c-4164-9b8a-a0be7721f99a service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:16.855608 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1398/2780] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:20:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:20:17.475759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f669b6-2632-47b8-b0d5-807522427842 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:17.477746 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-72f669b6-2632-47b8-b0d5-807522427842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:17.492555 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f669b6-2632-47b8-b0d5-807522427842 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:17.499904 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3177b4-20aa-423c-afc3-82abb28ea8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:17.500572 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3177b4-20aa-423c-afc3-82abb28ea8ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:17.500910 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3177b4-20aa-423c-afc3-82abb28ea8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:17.501201 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3177b4-20aa-423c-afc3-82abb28ea8ef admin admin] PATH_INFO: `/v3/projects/c736382d431f43e6910e7f69601e2c3e` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:17.502557 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3177b4-20aa-423c-afc3-82abb28ea8ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=c736382d431f43e6910e7f69601e2c3e)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:17.504751 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3177b4-20aa-423c-afc3-82abb28ea8ef admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:17.529301 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:20:17.632041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-da3177b4-20aa-423c-afc3-82abb28ea8ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c736382d431f43e6910e7f69601e2c3e', 'request_id': 'req-da3177b4-20aa-423c-afc3-82abb28ea8ef'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:17.639590 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-da3177b4-20aa-423c-afc3-82abb28ea8ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c736382d431f43e6910e7f69601e2c3e', 'request_id': 'req-da3177b4-20aa-423c-afc3-82abb28ea8ef'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:17.646982 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-da3177b4-20aa-423c-afc3-82abb28ea8ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c736382d431f43e6910e7f69601e2c3e', 'request_id': 'req-da3177b4-20aa-423c-afc3-82abb28ea8ef'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:17.709440 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-da3177b4-20aa-423c-afc3-82abb28ea8ef admin admin] The token cache is being invalidate because project c736382d431f43e6910e7f69601e2c3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:20:17.709730 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-da3177b4-20aa-423c-afc3-82abb28ea8ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:17.710445 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-da3177b4-20aa-423c-afc3-82abb28ea8ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:17.712637 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1383/2781] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:20:17 2022] DELETE /identity/v3/projects/c736382d431f43e6910e7f69601e2c3e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:20:17.717536 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe32f3-fc6d-448a-ae44-9db2aec4787f None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:17.718060 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe32f3-fc6d-448a-ae44-9db2aec4787f None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:17.718164 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbe32f3-fc6d-448a-ae44-9db2aec4787f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:17.738554 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9fbe32f3-fc6d-448a-ae44-9db2aec4787f None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:20:17.739752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9fbe32f3-fc6d-448a-ae44-9db2aec4787f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:17.800849 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1399/2782] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:20:17.810442 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bd2ca-7582-429f-8cef-aa92a89bc03e None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:17.810760 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bd2ca-7582-429f-8cef-aa92a89bc03e None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:17.811116 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2bd2ca-7582-429f-8cef-aa92a89bc03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:17.832442 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-9e2bd2ca-7582-429f-8cef-aa92a89bc03e None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:20:17.833677 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-9e2bd2ca-7582-429f-8cef-aa92a89bc03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:17.842466 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1384/2783] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:20:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:20:17.846697 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe32f3-fc6d-448a-ae44-9db2aec4787f None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:17.858118 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbe32f3-fc6d-448a-ae44-9db2aec4787f None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:17.866642 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8b1ec-b4b9-46d7-8d95-af2f38fe8822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:17.867524 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8b1ec-b4b9-46d7-8d95-af2f38fe8822 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:17.868411 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8b1ec-b4b9-46d7-8d95-af2f38fe8822 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:17.868775 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8b1ec-b4b9-46d7-8d95-af2f38fe8822 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:17.869581 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8b1ec-b4b9-46d7-8d95-af2f38fe8822 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:20:17.870002 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8b1ec-b4b9-46d7-8d95-af2f38fe8822 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:17.871683 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8b1ec-b4b9-46d7-8d95-af2f38fe8822 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:17.885388 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1400/2784] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:20:17.889527 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2bd2ca-7582-429f-8cef-aa92a89bc03e None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:17.898859 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2bd2ca-7582-429f-8cef-aa92a89bc03e None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:17.903865 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f26e8-b645-4d44-801d-16a70a77539e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:17.904485 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f26e8-b645-4d44-801d-16a70a77539e admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:17.904773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f26e8-b645-4d44-801d-16a70a77539e admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:17.905056 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f26e8-b645-4d44-801d-16a70a77539e admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:17.905727 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f26e8-b645-4d44-801d-16a70a77539e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:17.907702 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f26e8-b645-4d44-801d-16a70a77539e admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:17.997607 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1385/2785] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:20:17 2022] POST /identity/v3/projects => generated 397 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:20:18.001772 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8b1ec-b4b9-46d7-8d95-af2f38fe8822 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:18.012445 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8b1ec-b4b9-46d7-8d95-af2f38fe8822 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:18.049146 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:20:18.088803 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38597f-656f-4a15-be24-f12c5894bd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:18.089474 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38597f-656f-4a15-be24-f12c5894bd2d admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:18.089756 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38597f-656f-4a15-be24-f12c5894bd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:18.090064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38597f-656f-4a15-be24-f12c5894bd2d admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:18.090737 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38597f-656f-4a15-be24-f12c5894bd2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:18.092689 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b38597f-656f-4a15-be24-f12c5894bd2d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:18.198037 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1401/2786] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:20:18 2022] POST /identity/v3/users => generated 299 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:20:18.202290 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f26e8-b645-4d44-801d-16a70a77539e admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:18.214182 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f26e8-b645-4d44-801d-16a70a77539e admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:18.222006 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087306d-1845-46ac-8ac4-7b56d6610303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:18.223277 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087306d-1845-46ac-8ac4-7b56d6610303 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:18.224093 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087306d-1845-46ac-8ac4-7b56d6610303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:18.224556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087306d-1845-46ac-8ac4-7b56d6610303 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:18.225399 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b087306d-1845-46ac-8ac4-7b56d6610303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:18.228957 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b087306d-1845-46ac-8ac4-7b56d6610303 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:18.244884 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1386/2787] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:20:18.248781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38597f-656f-4a15-be24-f12c5894bd2d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:18.261674 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38597f-656f-4a15-be24-f12c5894bd2d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:18.269789 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326c481c-5758-4e6a-b335-b70842fe34cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:18.270767 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c481c-5758-4e6a-b335-b70842fe34cb admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:18.271146 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c481c-5758-4e6a-b335-b70842fe34cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:18.274750 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326c481c-5758-4e6a-b335-b70842fe34cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:18.274750 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c481c-5758-4e6a-b335-b70842fe34cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:18.274750 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326c481c-5758-4e6a-b335-b70842fe34cb admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:18.286229 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1402/2788] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:20:18.290878 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087306d-1845-46ac-8ac4-7b56d6610303 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:18.301496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b087306d-1845-46ac-8ac4-7b56d6610303 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:18.307566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62810445-a678-4eb3-80ce-14eac167a3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:18.308443 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62810445-a678-4eb3-80ce-14eac167a3c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:18.308822 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62810445-a678-4eb3-80ce-14eac167a3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:18.309199 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62810445-a678-4eb3-80ce-14eac167a3c2 admin admin] PATH_INFO: `/v3/projects/93837a125b7b4b8f81454c335e8c068c/users/65205e05eda14193969661bad00f9b2c/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:18.327337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62810445-a678-4eb3-80ce-14eac167a3c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93837a125b7b4b8f81454c335e8c068c, user_id=65205e05eda14193969661bad00f9b2c, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:18.330117 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62810445-a678-4eb3-80ce-14eac167a3c2 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:18.418196 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1387/2789] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:20:18 2022] PUT /identity/v3/projects/93837a125b7b4b8f81454c335e8c068c/users/65205e05eda14193969661bad00f9b2c/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:20:18.429203 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1e6f0-0d60-4f78-8964-1145f63a3ca6 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:18.429435 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1e6f0-0d60-4f78-8964-1145f63a3ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:18.429871 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa1e6f0-0d60-4f78-8964-1145f63a3ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:18.461312 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-bfa1e6f0-0d60-4f78-8964-1145f63a3ca6 None None] MFA Rules not processed for user `65205e05eda14193969661bad00f9b2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:20:18.462653 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bfa1e6f0-0d60-4f78-8964-1145f63a3ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:18.551121 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1403/2790] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:20:18 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:20:18.563871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62810445-a678-4eb3-80ce-14eac167a3c2 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:18.566007 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-62810445-a678-4eb3-80ce-14eac167a3c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:18.626444 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62810445-a678-4eb3-80ce-14eac167a3c2 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:18.630261 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d402b-0a23-482a-a061-656cc22f59df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:18.630875 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d402b-0a23-482a-a061-656cc22f59df service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:18.631175 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d402b-0a23-482a-a061-656cc22f59df service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:18.631446 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d402b-0a23-482a-a061-656cc22f59df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:18.645498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d402b-0a23-482a-a061-656cc22f59df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:18.647002 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d402b-0a23-482a-a061-656cc22f59df service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:18.660108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1388/2791] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:20:18 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:20:20.834391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1e6f0-0d60-4f78-8964-1145f63a3ca6 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:20.835990 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bfa1e6f0-0d60-4f78-8964-1145f63a3ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:20.917109 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa1e6f0-0d60-4f78-8964-1145f63a3ca6 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:20.922921 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37eed2c-aa31-411f-a0c8-decebca8d156 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad5cc9927374836b0cc74db85b686fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:20.923561 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37eed2c-aa31-411f-a0c8-decebca8d156 service placement] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:20.924202 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37eed2c-aa31-411f-a0c8-decebca8d156 service placement] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:20.924202 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37eed2c-aa31-411f-a0c8-decebca8d156 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:20.926424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f37eed2c-aa31-411f-a0c8-decebca8d156 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:20.940937 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37eed2c-aa31-411f-a0c8-decebca8d156 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:20.941978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37eed2c-aa31-411f-a0c8-decebca8d156 service placement] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:20.952975 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1404/2792] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:20:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:20:23.526960 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d402b-0a23-482a-a061-656cc22f59df service cinder] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:23.528373 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-ee4d402b-0a23-482a-a061-656cc22f59df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:23.574814 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:20:23.587087 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d402b-0a23-482a-a061-656cc22f59df service cinder] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:23.590944 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28baadff-6d53-4aa6-aa2a-b07c3b517dcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:23.595008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28baadff-6d53-4aa6-aa2a-b07c3b517dcc service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:23.595008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28baadff-6d53-4aa6-aa2a-b07c3b517dcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:23.595008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28baadff-6d53-4aa6-aa2a-b07c3b517dcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:23.595008 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-28baadff-6d53-4aa6-aa2a-b07c3b517dcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:23.672946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28baadff-6d53-4aa6-aa2a-b07c3b517dcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:23.674785 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28baadff-6d53-4aa6-aa2a-b07c3b517dcc service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:23.688459 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1389/2793] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:20:23 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:20:25.230305 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37eed2c-aa31-411f-a0c8-decebca8d156 service placement] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:25.231952 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f37eed2c-aa31-411f-a0c8-decebca8d156 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:25.281978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:20:25.309182 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37eed2c-aa31-411f-a0c8-decebca8d156 service placement] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:25.315713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67278a0a-239d-4c60-b5aa-c51435561086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:25.316606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67278a0a-239d-4c60-b5aa-c51435561086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:25.317006 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67278a0a-239d-4c60-b5aa-c51435561086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:25.317402 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67278a0a-239d-4c60-b5aa-c51435561086 admin admin] PATH_INFO: `/v3/users/a1300b00c7214d168e72850a8585bd47` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:25.319130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67278a0a-239d-4c60-b5aa-c51435561086 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1300b00c7214d168e72850a8585bd47)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:25.321633 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67278a0a-239d-4c60-b5aa-c51435561086 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:25.519307 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-67278a0a-239d-4c60-b5aa-c51435561086 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1300b00c7214d168e72850a8585bd47'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:25.526957 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-67278a0a-239d-4c60-b5aa-c51435561086 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1300b00c7214d168e72850a8585bd47'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:25.527647 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-67278a0a-239d-4c60-b5aa-c51435561086 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1300b00c7214d168e72850a8585bd47'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:25.540681 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-67278a0a-239d-4c60-b5aa-c51435561086 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1300b00c7214d168e72850a8585bd47'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:25.540893 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-67278a0a-239d-4c60-b5aa-c51435561086 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1300b00c7214d168e72850a8585bd47'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:25.635831 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1405/2794] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:20:25 2022] DELETE /identity/v3/users/a1300b00c7214d168e72850a8585bd47 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:20:25.960481 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28baadff-6d53-4aa6-aa2a-b07c3b517dcc service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:25.970918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28baadff-6d53-4aa6-aa2a-b07c3b517dcc service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:26.032828 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724323d9-2796-46ac-81f7-bc4e58a36568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:26.033483 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724323d9-2796-46ac-81f7-bc4e58a36568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:26.033764 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724323d9-2796-46ac-81f7-bc4e58a36568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:26.034076 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724323d9-2796-46ac-81f7-bc4e58a36568 admin admin] PATH_INFO: `/v3/projects/142acfd7d2a7483083b9807280598eef` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:26.035376 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724323d9-2796-46ac-81f7-bc4e58a36568 admin admin] RBAC: Authorizing `identity:delete_project(project_id=142acfd7d2a7483083b9807280598eef)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:26.037426 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724323d9-2796-46ac-81f7-bc4e58a36568 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:26.135273 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-724323d9-2796-46ac-81f7-bc4e58a36568 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '142acfd7d2a7483083b9807280598eef', 'request_id': 'req-724323d9-2796-46ac-81f7-bc4e58a36568'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:26.141631 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-724323d9-2796-46ac-81f7-bc4e58a36568 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '142acfd7d2a7483083b9807280598eef', 'request_id': 'req-724323d9-2796-46ac-81f7-bc4e58a36568'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:26.148041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-724323d9-2796-46ac-81f7-bc4e58a36568 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '142acfd7d2a7483083b9807280598eef', 'request_id': 'req-724323d9-2796-46ac-81f7-bc4e58a36568'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:26.201081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-724323d9-2796-46ac-81f7-bc4e58a36568 admin admin] The token cache is being invalidate because project 142acfd7d2a7483083b9807280598eef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:20:26.201443 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-724323d9-2796-46ac-81f7-bc4e58a36568 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:26.202035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-724323d9-2796-46ac-81f7-bc4e58a36568 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:26.203813 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1390/2795] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:20:25 2022] DELETE /identity/v3/projects/142acfd7d2a7483083b9807280598eef => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:20:26.207811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67278a0a-239d-4c60-b5aa-c51435561086 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:26.209259 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-67278a0a-239d-4c60-b5aa-c51435561086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:26.222714 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67278a0a-239d-4c60-b5aa-c51435561086 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:26.281838 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2708b50-c5f5-4781-a830-ba0113741d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:26.282529 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2708b50-c5f5-4781-a830-ba0113741d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:26.282812 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2708b50-c5f5-4781-a830-ba0113741d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:26.283091 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2708b50-c5f5-4781-a830-ba0113741d05 admin admin] PATH_INFO: `/v3/users/8bf4fb63bfc34115a69917d23d68c659` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:26.284294 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2708b50-c5f5-4781-a830-ba0113741d05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bf4fb63bfc34115a69917d23d68c659)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:26.286283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2708b50-c5f5-4781-a830-ba0113741d05 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:26.518744 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b2708b50-c5f5-4781-a830-ba0113741d05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bf4fb63bfc34115a69917d23d68c659'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:26.525816 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b2708b50-c5f5-4781-a830-ba0113741d05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bf4fb63bfc34115a69917d23d68c659'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:26.526549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b2708b50-c5f5-4781-a830-ba0113741d05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bf4fb63bfc34115a69917d23d68c659'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:26.529956 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:20:26.540381 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b2708b50-c5f5-4781-a830-ba0113741d05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bf4fb63bfc34115a69917d23d68c659'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:26.541088 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b2708b50-c5f5-4781-a830-ba0113741d05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bf4fb63bfc34115a69917d23d68c659'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:26.624784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1406/2796] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:20:26 2022] DELETE /identity/v3/users/8bf4fb63bfc34115a69917d23d68c659 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:20:26.864230 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724323d9-2796-46ac-81f7-bc4e58a36568 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:26.865538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-724323d9-2796-46ac-81f7-bc4e58a36568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:26.879843 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724323d9-2796-46ac-81f7-bc4e58a36568 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:26.909448 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:20:26.954421 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310eddf7-31ef-40b1-8c17-d9af73151037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:26.955083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310eddf7-31ef-40b1-8c17-d9af73151037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:26.955425 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310eddf7-31ef-40b1-8c17-d9af73151037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:26.955709 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310eddf7-31ef-40b1-8c17-d9af73151037 admin admin] PATH_INFO: `/v3/projects/1cb9ed01f9f74a1c90b1ab967e9a9607` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:26.956907 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310eddf7-31ef-40b1-8c17-d9af73151037 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cb9ed01f9f74a1c90b1ab967e9a9607)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:26.958930 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310eddf7-31ef-40b1-8c17-d9af73151037 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:27.086776 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-310eddf7-31ef-40b1-8c17-d9af73151037 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cb9ed01f9f74a1c90b1ab967e9a9607', 'request_id': 'req-310eddf7-31ef-40b1-8c17-d9af73151037'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:27.094080 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-310eddf7-31ef-40b1-8c17-d9af73151037 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cb9ed01f9f74a1c90b1ab967e9a9607', 'request_id': 'req-310eddf7-31ef-40b1-8c17-d9af73151037'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:27.100415 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-310eddf7-31ef-40b1-8c17-d9af73151037 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cb9ed01f9f74a1c90b1ab967e9a9607', 'request_id': 'req-310eddf7-31ef-40b1-8c17-d9af73151037'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:27.176867 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-310eddf7-31ef-40b1-8c17-d9af73151037 admin admin] The token cache is being invalidate because project 1cb9ed01f9f74a1c90b1ab967e9a9607 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:20:27.177157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-310eddf7-31ef-40b1-8c17-d9af73151037 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:27.177763 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-310eddf7-31ef-40b1-8c17-d9af73151037 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:27.179306 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1391/2797] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:20:26 2022] DELETE /identity/v3/projects/1cb9ed01f9f74a1c90b1ab967e9a9607 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:20:30.876263 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2708b50-c5f5-4781-a830-ba0113741d05 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:30.877667 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b2708b50-c5f5-4781-a830-ba0113741d05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:30.890529 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2708b50-c5f5-4781-a830-ba0113741d05 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:30.948006 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7711b-7e13-42f8-9e2d-e9c1d21e56c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:30.948650 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7711b-7e13-42f8-9e2d-e9c1d21e56c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:30.948927 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7711b-7e13-42f8-9e2d-e9c1d21e56c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:30.949200 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb7711b-7e13-42f8-9e2d-e9c1d21e56c0 admin admin] PATH_INFO: `/v3/users/65205e05eda14193969661bad00f9b2c` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:30.950400 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb7711b-7e13-42f8-9e2d-e9c1d21e56c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65205e05eda14193969661bad00f9b2c)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:30.952418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb7711b-7e13-42f8-9e2d-e9c1d21e56c0 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:31.151392 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-bdb7711b-7e13-42f8-9e2d-e9c1d21e56c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65205e05eda14193969661bad00f9b2c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:31.158237 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-bdb7711b-7e13-42f8-9e2d-e9c1d21e56c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65205e05eda14193969661bad00f9b2c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:31.159019 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-bdb7711b-7e13-42f8-9e2d-e9c1d21e56c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65205e05eda14193969661bad00f9b2c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:31.171729 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-bdb7711b-7e13-42f8-9e2d-e9c1d21e56c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65205e05eda14193969661bad00f9b2c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:31.172365 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-bdb7711b-7e13-42f8-9e2d-e9c1d21e56c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65205e05eda14193969661bad00f9b2c'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:31.251983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1407/2798] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:20:30 2022] DELETE /identity/v3/users/65205e05eda14193969661bad00f9b2c => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:20:31.256566 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056b652-fd2f-40df-b8a2-d1698aefbadb None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:31.257087 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056b652-fd2f-40df-b8a2-d1698aefbadb None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:31.257087 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056b652-fd2f-40df-b8a2-d1698aefbadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:31.276321 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-c056b652-fd2f-40df-b8a2-d1698aefbadb None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:20:31.277501 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-c056b652-fd2f-40df-b8a2-d1698aefbadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:31.337794 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1392/2799] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:20:31.347586 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7711b-7e13-42f8-9e2d-e9c1d21e56c0 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:31.349005 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-bdb7711b-7e13-42f8-9e2d-e9c1d21e56c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:31.406859 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb7711b-7e13-42f8-9e2d-e9c1d21e56c0 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:31.410542 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e864744-e3be-4f13-bbcd-3c9ea4ffbd8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:31.411152 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e864744-e3be-4f13-bbcd-3c9ea4ffbd8c service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:31.411443 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e864744-e3be-4f13-bbcd-3c9ea4ffbd8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:31.411724 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e864744-e3be-4f13-bbcd-3c9ea4ffbd8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:31.419218 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:20:31.421461 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e864744-e3be-4f13-bbcd-3c9ea4ffbd8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:31.422935 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e864744-e3be-4f13-bbcd-3c9ea4ffbd8c service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:31.433123 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1408/2800] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:20:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:20:31.829181 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056b652-fd2f-40df-b8a2-d1698aefbadb None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:31.830723 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-c056b652-fd2f-40df-b8a2-d1698aefbadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:31.842913 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056b652-fd2f-40df-b8a2-d1698aefbadb None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:31.847669 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32a5fe-a6e0-4bc0-a410-2bed03593dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:31.848320 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32a5fe-a6e0-4bc0-a410-2bed03593dee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:31.848610 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32a5fe-a6e0-4bc0-a410-2bed03593dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:31.848890 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32a5fe-a6e0-4bc0-a410-2bed03593dee admin admin] PATH_INFO: `/v3/projects/93837a125b7b4b8f81454c335e8c068c` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:31.850157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32a5fe-a6e0-4bc0-a410-2bed03593dee admin admin] RBAC: Authorizing `identity:delete_project(project_id=93837a125b7b4b8f81454c335e8c068c)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:31.852798 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a32a5fe-a6e0-4bc0-a410-2bed03593dee admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:31.964645 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:20:31.966296 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-7a32a5fe-a6e0-4bc0-a410-2bed03593dee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93837a125b7b4b8f81454c335e8c068c', 'request_id': 'req-7a32a5fe-a6e0-4bc0-a410-2bed03593dee'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:31.972901 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-7a32a5fe-a6e0-4bc0-a410-2bed03593dee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93837a125b7b4b8f81454c335e8c068c', 'request_id': 'req-7a32a5fe-a6e0-4bc0-a410-2bed03593dee'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:31.979263 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-7a32a5fe-a6e0-4bc0-a410-2bed03593dee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93837a125b7b4b8f81454c335e8c068c', 'request_id': 'req-7a32a5fe-a6e0-4bc0-a410-2bed03593dee'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:32.061652 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-7a32a5fe-a6e0-4bc0-a410-2bed03593dee admin admin] The token cache is being invalidate because project 93837a125b7b4b8f81454c335e8c068c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:20:32.061987 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-7a32a5fe-a6e0-4bc0-a410-2bed03593dee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:32.062619 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-7a32a5fe-a6e0-4bc0-a410-2bed03593dee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:20:32.064138 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1393/2801] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:20:31 2022] DELETE /identity/v3/projects/93837a125b7b4b8f81454c335e8c068c => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:20:41.560065 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=2 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:20:42.136910 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=7,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:20:52.193852 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e864744-e3be-4f13-bbcd-3c9ea4ffbd8c service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:20:52.195708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-5e864744-e3be-4f13-bbcd-3c9ea4ffbd8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:52.267982 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e864744-e3be-4f13-bbcd-3c9ea4ffbd8c service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:20:52.329356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e04fc-533c-4366-9550-e842cf10fe63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:20:52.330018 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e04fc-533c-4366-9550-e842cf10fe63 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:52.330293 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e04fc-533c-4366-9550-e842cf10fe63 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:52.330572 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e04fc-533c-4366-9550-e842cf10fe63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:52.332339 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3b7e04fc-533c-4366-9550-e842cf10fe63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:52.388781 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e04fc-533c-4366-9550-e842cf10fe63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:20:52.390447 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7e04fc-533c-4366-9550-e842cf10fe63 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:20:52.399938 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1409/2802] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:20:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:20:58.951324 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697e038-6538-4522-94bb-c962cee88f67 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:20:58.951858 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697e038-6538-4522-94bb-c962cee88f67 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:20:58.951974 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d697e038-6538-4522-94bb-c962cee88f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:20:58.973054 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-d697e038-6538-4522-94bb-c962cee88f67 None None] MFA Rules not processed for user `77aff587ea964860895cd4dce03f2158`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:20:58.974983 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-d697e038-6538-4522-94bb-c962cee88f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:20:58.982895 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1394/2803] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:20:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:21:02.395273 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=84 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:21:08.966275 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:22:20.950489 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e04fc-533c-4366-9550-e842cf10fe63 service nova] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:22:20.952000 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3b7e04fc-533c-4366-9550-e842cf10fe63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:22:21.008228 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7e04fc-533c-4366-9550-e842cf10fe63 service nova] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:22:21.011811 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f67fa-9990-43d4-8e9d-77d7906ff129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:22:21.012424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f67fa-9990-43d4-8e9d-77d7906ff129 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:22:21.012706 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f67fa-9990-43d4-8e9d-77d7906ff129 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:22:21.012983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f67fa-9990-43d4-8e9d-77d7906ff129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:22:21.014733 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b02f67fa-9990-43d4-8e9d-77d7906ff129 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:22:21.023977 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f67fa-9990-43d4-8e9d-77d7906ff129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:22:21.025367 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f67fa-9990-43d4-8e9d-77d7906ff129 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:22:21.033256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1410/2804] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:22:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:22:30.509905 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697e038-6538-4522-94bb-c962cee88f67 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:22:30.519902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d697e038-6538-4522-94bb-c962cee88f67 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:22:30.524352 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700bd37-47f1-43d8-8498-8b4f386c41e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:22:30.524969 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700bd37-47f1-43d8-8498-8b4f386c41e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:22:30.525255 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700bd37-47f1-43d8-8498-8b4f386c41e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:22:30.525535 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c700bd37-47f1-43d8-8498-8b4f386c41e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:22:30.527311 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-c700bd37-47f1-43d8-8498-8b4f386c41e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:22:30.541545 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c700bd37-47f1-43d8-8498-8b4f386c41e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:22:30.543043 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c700bd37-47f1-43d8-8498-8b4f386c41e3 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:22:30.556547 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1395/2805] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:22:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:22:31.029660 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=26 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:22:31.624264 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f67fa-9990-43d4-8e9d-77d7906ff129 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:22:31.625649 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b02f67fa-9990-43d4-8e9d-77d7906ff129 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:22:31.683495 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f67fa-9990-43d4-8e9d-77d7906ff129 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:22:31.696869 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755361f6-5e4d-4e79-8011-a351033853c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:22:31.696869 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755361f6-5e4d-4e79-8011-a351033853c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:22:31.696869 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755361f6-5e4d-4e79-8011-a351033853c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:22:31.696869 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755361f6-5e4d-4e79-8011-a351033853c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:22:31.698032 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-755361f6-5e4d-4e79-8011-a351033853c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:22:31.707825 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755361f6-5e4d-4e79-8011-a351033853c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:22:31.709282 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755361f6-5e4d-4e79-8011-a351033853c1 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:22:31.721659 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1411/2806] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:22:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:22:40.550777 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=6 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:22:41.715905 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=26 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:22:45.949961 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700bd37-47f1-43d8-8498-8b4f386c41e3 service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:22:45.951500 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-c700bd37-47f1-43d8-8498-8b4f386c41e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:22:46.012365 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c700bd37-47f1-43d8-8498-8b4f386c41e3 service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:22:46.016214 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78c5e5-0f89-4e97-bf79-72ff68714c48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:22:46.016824 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78c5e5-0f89-4e97-bf79-72ff68714c48 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:22:46.017109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78c5e5-0f89-4e97-bf79-72ff68714c48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:22:46.017386 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b78c5e5-0f89-4e97-bf79-72ff68714c48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:22:46.019152 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-5b78c5e5-0f89-4e97-bf79-72ff68714c48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:22:46.121990 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78c5e5-0f89-4e97-bf79-72ff68714c48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:22:46.123363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b78c5e5-0f89-4e97-bf79-72ff68714c48 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:22:46.142573 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1396/2807] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:22:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:22:53.288979 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755361f6-5e4d-4e79-8011-a351033853c1 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:22:53.290575 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-755361f6-5e4d-4e79-8011-a351033853c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:22:53.343624 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755361f6-5e4d-4e79-8011-a351033853c1 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:22:53.348244 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e414a-4760-4a97-9b24-07b99deeffc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:22:53.348881 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e414a-4760-4a97-9b24-07b99deeffc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:22:53.349162 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e414a-4760-4a97-9b24-07b99deeffc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:22:53.350022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7e414a-4760-4a97-9b24-07b99deeffc6 admin admin] PATH_INFO: `/v3/users/02569c995a6d4b678933599f60ad3909` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:22:53.351322 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e414a-4760-4a97-9b24-07b99deeffc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02569c995a6d4b678933599f60ad3909)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:22:53.353602 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7e414a-4760-4a97-9b24-07b99deeffc6 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:22:53.507927 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4f7e414a-4760-4a97-9b24-07b99deeffc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02569c995a6d4b678933599f60ad3909'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:22:53.517774 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4f7e414a-4760-4a97-9b24-07b99deeffc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02569c995a6d4b678933599f60ad3909'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:22:53.518515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4f7e414a-4760-4a97-9b24-07b99deeffc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02569c995a6d4b678933599f60ad3909'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:22:53.531333 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4f7e414a-4760-4a97-9b24-07b99deeffc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02569c995a6d4b678933599f60ad3909'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:22:53.532081 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-4f7e414a-4760-4a97-9b24-07b99deeffc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02569c995a6d4b678933599f60ad3909'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:22:53.618513 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1412/2808] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:22:53 2022] DELETE /identity/v3/users/02569c995a6d4b678933599f60ad3909 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:22:54.008290 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78c5e5-0f89-4e97-bf79-72ff68714c48 service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:22:54.021197 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b78c5e5-0f89-4e97-bf79-72ff68714c48 service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:22:54.072670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:22:54.078596 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376548d1-1c95-440d-a22f-8965a429d308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:22:54.087752 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376548d1-1c95-440d-a22f-8965a429d308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:22:54.087752 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376548d1-1c95-440d-a22f-8965a429d308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:22:54.087752 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376548d1-1c95-440d-a22f-8965a429d308 admin admin] PATH_INFO: `/v3/projects/d65888d17d2e439592600a4e9305eeed` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:22:54.087752 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376548d1-1c95-440d-a22f-8965a429d308 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d65888d17d2e439592600a4e9305eeed)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:22:54.087752 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376548d1-1c95-440d-a22f-8965a429d308 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:22:54.241037 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-376548d1-1c95-440d-a22f-8965a429d308 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd65888d17d2e439592600a4e9305eeed', 'request_id': 'req-376548d1-1c95-440d-a22f-8965a429d308'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:22:54.247103 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-376548d1-1c95-440d-a22f-8965a429d308 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd65888d17d2e439592600a4e9305eeed', 'request_id': 'req-376548d1-1c95-440d-a22f-8965a429d308'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:22:54.252667 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-376548d1-1c95-440d-a22f-8965a429d308 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd65888d17d2e439592600a4e9305eeed', 'request_id': 'req-376548d1-1c95-440d-a22f-8965a429d308'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:22:54.326109 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-376548d1-1c95-440d-a22f-8965a429d308 admin admin] The token cache is being invalidate because project d65888d17d2e439592600a4e9305eeed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:22:54.326405 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-376548d1-1c95-440d-a22f-8965a429d308 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:22:54.327028 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-376548d1-1c95-440d-a22f-8965a429d308 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:22:54.328556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1397/2809] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:22:54 2022] DELETE /identity/v3/projects/d65888d17d2e439592600a4e9305eeed => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:22:54.332732 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e414a-4760-4a97-9b24-07b99deeffc6 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:22:54.334179 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4f7e414a-4760-4a97-9b24-07b99deeffc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:22:54.347494 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7e414a-4760-4a97-9b24-07b99deeffc6 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:22:54.402976 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7acf0-4fb7-4699-8147-c817dcc88fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:22:54.403612 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7acf0-4fb7-4699-8147-c817dcc88fa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:22:54.403895 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7acf0-4fb7-4699-8147-c817dcc88fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:22:54.404172 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7acf0-4fb7-4699-8147-c817dcc88fa2 admin admin] PATH_INFO: `/v3/users/5077255a76fc45d9a62557db042932da` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:22:54.405325 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f7acf0-4fb7-4699-8147-c817dcc88fa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5077255a76fc45d9a62557db042932da)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:22:54.407273 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f7acf0-4fb7-4699-8147-c817dcc88fa2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:22:54.609867 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:22:54.614669 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b5f7acf0-4fb7-4699-8147-c817dcc88fa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5077255a76fc45d9a62557db042932da'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:22:54.621917 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b5f7acf0-4fb7-4699-8147-c817dcc88fa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5077255a76fc45d9a62557db042932da'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:22:54.622591 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b5f7acf0-4fb7-4699-8147-c817dcc88fa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5077255a76fc45d9a62557db042932da'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:22:54.634924 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b5f7acf0-4fb7-4699-8147-c817dcc88fa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5077255a76fc45d9a62557db042932da'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:22:54.635556 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-b5f7acf0-4fb7-4699-8147-c817dcc88fa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5077255a76fc45d9a62557db042932da'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:22:54.686909 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1413/2810] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:22:54 2022] DELETE /identity/v3/users/5077255a76fc45d9a62557db042932da => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:22:54.941783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376548d1-1c95-440d-a22f-8965a429d308 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:22:54.945459 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-376548d1-1c95-440d-a22f-8965a429d308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:22:54.966662 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376548d1-1c95-440d-a22f-8965a429d308 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:22:55.033531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ad839-153b-43a0-a62d-4223aef9c59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:22:55.034272 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ad839-153b-43a0-a62d-4223aef9c59d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:22:55.034483 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ad839-153b-43a0-a62d-4223aef9c59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:22:55.034779 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ad839-153b-43a0-a62d-4223aef9c59d admin admin] PATH_INFO: `/v3/projects/8fd19ae801584f7e899b3b3fb2a36136` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:22:55.037524 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ad839-153b-43a0-a62d-4223aef9c59d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fd19ae801584f7e899b3b3fb2a36136)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:22:55.038932 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ad839-153b-43a0-a62d-4223aef9c59d admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:22:55.157513 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8d6ad839-153b-43a0-a62d-4223aef9c59d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fd19ae801584f7e899b3b3fb2a36136', 'request_id': 'req-8d6ad839-153b-43a0-a62d-4223aef9c59d'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:22:55.164902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8d6ad839-153b-43a0-a62d-4223aef9c59d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fd19ae801584f7e899b3b3fb2a36136', 'request_id': 'req-8d6ad839-153b-43a0-a62d-4223aef9c59d'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:22:55.171189 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8d6ad839-153b-43a0-a62d-4223aef9c59d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fd19ae801584f7e899b3b3fb2a36136', 'request_id': 'req-8d6ad839-153b-43a0-a62d-4223aef9c59d'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:22:55.246775 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8d6ad839-153b-43a0-a62d-4223aef9c59d admin admin] The token cache is being invalidate because project 8fd19ae801584f7e899b3b3fb2a36136 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:22:55.247063 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8d6ad839-153b-43a0-a62d-4223aef9c59d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:22:55.247678 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8d6ad839-153b-43a0-a62d-4223aef9c59d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:22:55.249189 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1398/2811] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:22:54 2022] DELETE /identity/v3/projects/8fd19ae801584f7e899b3b3fb2a36136 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:01.111190 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7acf0-4fb7-4699-8147-c817dcc88fa2 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:01.112609 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-b5f7acf0-4fb7-4699-8147-c817dcc88fa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:01.192310 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7acf0-4fb7-4699-8147-c817dcc88fa2 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:01.266451 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a1989-b835-4e66-ad0f-e79df52f0377 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3191d804e8dc4b1a85f490319833ec07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:01.267258 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a1989-b835-4e66-ad0f-e79df52f0377 service nova] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:01.267543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a1989-b835-4e66-ad0f-e79df52f0377 service nova] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:01.267850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112a1989-b835-4e66-ad0f-e79df52f0377 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:01.269755 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-112a1989-b835-4e66-ad0f-e79df52f0377 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:01.289850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a1989-b835-4e66-ad0f-e79df52f0377 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:01.289850 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112a1989-b835-4e66-ad0f-e79df52f0377 service nova] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:01.297369 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1414/2812] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:23:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:23:05.243556 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=76,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:07.526461 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ad839-153b-43a0-a62d-4223aef9c59d admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:07.527872 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-8d6ad839-153b-43a0-a62d-4223aef9c59d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:07.584524 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ad839-153b-43a0-a62d-4223aef9c59d admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:07.588671 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4c4f3-badb-4b0a-8bc3-b7617dbbab08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:07.589306 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4c4f3-badb-4b0a-8bc3-b7617dbbab08 service neutron] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:07.589592 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4c4f3-badb-4b0a-8bc3-b7617dbbab08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:07.589871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4c4f3-badb-4b0a-8bc3-b7617dbbab08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:07.591625 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-a0e4c4f3-badb-4b0a-8bc3-b7617dbbab08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:07.652828 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e4c4f3-badb-4b0a-8bc3-b7617dbbab08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:07.654288 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e4c4f3-badb-4b0a-8bc3-b7617dbbab08 service neutron] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:07.663902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1399/2813] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:23:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:23:11.267949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a1989-b835-4e66-ad0f-e79df52f0377 service nova] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:11.269357 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-112a1989-b835-4e66-ad0f-e79df52f0377 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:11.285022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112a1989-b835-4e66-ad0f-e79df52f0377 service nova] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:11.289829 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266ee66-e2ce-4d8c-a632-b18381b8ff00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:11.290525 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266ee66-e2ce-4d8c-a632-b18381b8ff00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:11.290815 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266ee66-e2ce-4d8c-a632-b18381b8ff00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:11.291091 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266ee66-e2ce-4d8c-a632-b18381b8ff00 admin admin] PATH_INFO: `/v3/users/77aff587ea964860895cd4dce03f2158` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:11.292285 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266ee66-e2ce-4d8c-a632-b18381b8ff00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77aff587ea964860895cd4dce03f2158)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:11.294672 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266ee66-e2ce-4d8c-a632-b18381b8ff00 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:11.457264 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:11.465413 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-7266ee66-e2ce-4d8c-a632-b18381b8ff00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77aff587ea964860895cd4dce03f2158'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:11.472280 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-7266ee66-e2ce-4d8c-a632-b18381b8ff00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77aff587ea964860895cd4dce03f2158'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:11.472946 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-7266ee66-e2ce-4d8c-a632-b18381b8ff00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77aff587ea964860895cd4dce03f2158'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:11.485055 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-7266ee66-e2ce-4d8c-a632-b18381b8ff00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77aff587ea964860895cd4dce03f2158'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:11.485691 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-7266ee66-e2ce-4d8c-a632-b18381b8ff00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77aff587ea964860895cd4dce03f2158'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:11.549109 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1415/2814] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:23:11 2022] DELETE /identity/v3/users/77aff587ea964860895cd4dce03f2158 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:11.774310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4c4f3-badb-4b0a-8bc3-b7617dbbab08 service neutron] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:11.786154 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4c4f3-badb-4b0a-8bc3-b7617dbbab08 service neutron] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:11.843866 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0779351-43cb-4866-a72a-608988c7880b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:11.844531 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0779351-43cb-4866-a72a-608988c7880b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:11.844808 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0779351-43cb-4866-a72a-608988c7880b admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:11.845081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0779351-43cb-4866-a72a-608988c7880b admin admin] PATH_INFO: `/v3/projects/bfaca149a97a44cf84f9d21fa65510b6` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:11.846314 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0779351-43cb-4866-a72a-608988c7880b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfaca149a97a44cf84f9d21fa65510b6)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:11.848208 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0779351-43cb-4866-a72a-608988c7880b admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:11.972870 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-f0779351-43cb-4866-a72a-608988c7880b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfaca149a97a44cf84f9d21fa65510b6', 'request_id': 'req-f0779351-43cb-4866-a72a-608988c7880b'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:11.974784 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:11.979734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-f0779351-43cb-4866-a72a-608988c7880b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfaca149a97a44cf84f9d21fa65510b6', 'request_id': 'req-f0779351-43cb-4866-a72a-608988c7880b'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:11.985319 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-f0779351-43cb-4866-a72a-608988c7880b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfaca149a97a44cf84f9d21fa65510b6', 'request_id': 'req-f0779351-43cb-4866-a72a-608988c7880b'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:12.045178 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-f0779351-43cb-4866-a72a-608988c7880b admin admin] The token cache is being invalidate because project bfaca149a97a44cf84f9d21fa65510b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:23:12.045577 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-f0779351-43cb-4866-a72a-608988c7880b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:12.046663 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-f0779351-43cb-4866-a72a-608988c7880b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:12.048713 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1400/2815] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:23:11 2022] DELETE /identity/v3/projects/bfaca149a97a44cf84f9d21fa65510b6 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:12.053409 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37348eed-0f4e-4f06-8ab6-6680a6a7f891 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:12.053702 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37348eed-0f4e-4f06-8ab6-6680a6a7f891 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:12.054015 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37348eed-0f4e-4f06-8ab6-6680a6a7f891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:12.110987 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-37348eed-0f4e-4f06-8ab6-6680a6a7f891 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:23:12.112208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-37348eed-0f4e-4f06-8ab6-6680a6a7f891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:12.169855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1416/2816] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:23:12.177776 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f1f55-1c00-4c68-bd42-45af6bb50386 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:12.178342 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f1f55-1c00-4c68-bd42-45af6bb50386 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:12.178626 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f1f55-1c00-4c68-bd42-45af6bb50386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:12.199619 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-398f1f55-1c00-4c68-bd42-45af6bb50386 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:23:12.200817 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-398f1f55-1c00-4c68-bd42-45af6bb50386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:12.208759 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1401/2817] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:23:12.213372 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37348eed-0f4e-4f06-8ab6-6680a6a7f891 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:12.222992 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37348eed-0f4e-4f06-8ab6-6680a6a7f891 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:12.227573 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da6ccf-ce6b-45b8-ae53-8fe2f4eb1ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:12.228181 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da6ccf-ce6b-45b8-ae53-8fe2f4eb1ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:12.228467 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da6ccf-ce6b-45b8-ae53-8fe2f4eb1ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:12.228746 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88da6ccf-ce6b-45b8-ae53-8fe2f4eb1ae3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:12.229442 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da6ccf-ce6b-45b8-ae53-8fe2f4eb1ae3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:23:12.229751 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da6ccf-ce6b-45b8-ae53-8fe2f4eb1ae3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:12.231459 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88da6ccf-ce6b-45b8-ae53-8fe2f4eb1ae3 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:12.242687 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1417/2818] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:23:12.246720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f1f55-1c00-4c68-bd42-45af6bb50386 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:12.255773 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f1f55-1c00-4c68-bd42-45af6bb50386 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:12.260917 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419866c4-d0f6-442e-b6f2-46f9a03f7846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:12.261514 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419866c4-d0f6-442e-b6f2-46f9a03f7846 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:12.261795 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419866c4-d0f6-442e-b6f2-46f9a03f7846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:12.262104 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419866c4-d0f6-442e-b6f2-46f9a03f7846 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:12.262770 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419866c4-d0f6-442e-b6f2-46f9a03f7846 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:12.264646 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419866c4-d0f6-442e-b6f2-46f9a03f7846 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:12.367997 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1402/2819] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:23:12 2022] POST /identity/v3/projects => generated 365 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:23:12.372030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da6ccf-ce6b-45b8-ae53-8fe2f4eb1ae3 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:12.382207 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88da6ccf-ce6b-45b8-ae53-8fe2f4eb1ae3 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:12.436863 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbcb1e-a86e-45d0-bd53-676095dd700d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:12.437469 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbcb1e-a86e-45d0-bd53-676095dd700d admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:12.437752 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbcb1e-a86e-45d0-bd53-676095dd700d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:12.438064 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbcb1e-a86e-45d0-bd53-676095dd700d admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:12.438726 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afbcb1e-a86e-45d0-bd53-676095dd700d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:12.440664 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afbcb1e-a86e-45d0-bd53-676095dd700d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:12.511140 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:12.516242 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1418/2820] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:23:12 2022] POST /identity/v3/users => generated 283 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:23:12.520705 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419866c4-d0f6-442e-b6f2-46f9a03f7846 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:12.531389 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419866c4-d0f6-442e-b6f2-46f9a03f7846 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:12.536020 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771f6d5-8bd9-4817-8e64-0937de4e3883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:12.536619 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771f6d5-8bd9-4817-8e64-0937de4e3883 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:12.536905 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771f6d5-8bd9-4817-8e64-0937de4e3883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:12.537218 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8771f6d5-8bd9-4817-8e64-0937de4e3883 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:12.537874 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8771f6d5-8bd9-4817-8e64-0937de4e3883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:12.539484 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8771f6d5-8bd9-4817-8e64-0937de4e3883 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:12.549274 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1403/2821] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:23:12.553402 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbcb1e-a86e-45d0-bd53-676095dd700d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:12.564191 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbcb1e-a86e-45d0-bd53-676095dd700d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:12.568984 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529d8c4-597d-48cd-9ff8-e93249744b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:12.569598 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529d8c4-597d-48cd-9ff8-e93249744b1a admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:12.570122 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529d8c4-597d-48cd-9ff8-e93249744b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:12.571248 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529d8c4-597d-48cd-9ff8-e93249744b1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:12.572249 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529d8c4-597d-48cd-9ff8-e93249744b1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:12.573837 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529d8c4-597d-48cd-9ff8-e93249744b1a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:12.585903 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1419/2822] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:23:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:23:12.590143 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771f6d5-8bd9-4817-8e64-0937de4e3883 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:12.600944 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8771f6d5-8bd9-4817-8e64-0937de4e3883 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:12.605572 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac551b-72ab-460e-bf38-a22dc1f72c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:12.606257 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac551b-72ab-460e-bf38-a22dc1f72c41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:12.606538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac551b-72ab-460e-bf38-a22dc1f72c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:12.606815 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac551b-72ab-460e-bf38-a22dc1f72c41 admin admin] PATH_INFO: `/v3/projects/79a7356630c34973a52df71de19dbc0a/users/0beb5f1b5ef949c4a7cbe9361a10a44a/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:12.622622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac551b-72ab-460e-bf38-a22dc1f72c41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79a7356630c34973a52df71de19dbc0a, user_id=0beb5f1b5ef949c4a7cbe9361a10a44a, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:12.625334 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac551b-72ab-460e-bf38-a22dc1f72c41 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:12.709192 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1404/2823] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:23:12 2022] PUT /identity/v3/projects/79a7356630c34973a52df71de19dbc0a/users/0beb5f1b5ef949c4a7cbe9361a10a44a/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:12.720741 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc606f-1d12-4864-8a04-3d9c46c0be18 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:12.721033 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc606f-1d12-4864-8a04-3d9c46c0be18 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:12.721316 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc606f-1d12-4864-8a04-3d9c46c0be18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:12.749828 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-08cc606f-1d12-4864-8a04-3d9c46c0be18 None None] MFA Rules not processed for user `0beb5f1b5ef949c4a7cbe9361a10a44a`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:23:12.751077 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-08cc606f-1d12-4864-8a04-3d9c46c0be18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:12.814668 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1420/2824] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:23:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:23:12.819116 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac551b-72ab-460e-bf38-a22dc1f72c41 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:12.880418 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac551b-72ab-460e-bf38-a22dc1f72c41 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:12.885037 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7af76-8ab4-495e-82c9-1436df1b8113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:12.885653 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7af76-8ab4-495e-82c9-1436df1b8113 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:12.885967 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7af76-8ab4-495e-82c9-1436df1b8113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:12.886249 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7af76-8ab4-495e-82c9-1436df1b8113 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:12.886926 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7af76-8ab4-495e-82c9-1436df1b8113 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:12.888827 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7af76-8ab4-495e-82c9-1436df1b8113 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:12.962503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1405/2825] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:23:12 2022] POST /identity/v3/projects => generated 363 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:23:12.966704 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc606f-1d12-4864-8a04-3d9c46c0be18 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:12.977868 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc606f-1d12-4864-8a04-3d9c46c0be18 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:13.034391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f926c-515a-4d52-b416-cc503ad75540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:13.035020 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f926c-515a-4d52-b416-cc503ad75540 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:13.035305 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f926c-515a-4d52-b416-cc503ad75540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:13.035581 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f926c-515a-4d52-b416-cc503ad75540 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:13.036254 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f926c-515a-4d52-b416-cc503ad75540 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:13.038192 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f926c-515a-4d52-b416-cc503ad75540 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:13.126547 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1421/2826] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:23:12 2022] POST /identity/v3/users => generated 288 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:23:13.132692 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7af76-8ab4-495e-82c9-1436df1b8113 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:13.142021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7af76-8ab4-495e-82c9-1436df1b8113 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:13.146823 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7c09e-ac9c-4386-acac-2106d84bebc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:13.147439 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7c09e-ac9c-4386-acac-2106d84bebc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:13.147720 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7c09e-ac9c-4386-acac-2106d84bebc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:13.147998 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7c09e-ac9c-4386-acac-2106d84bebc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:13.149223 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7c09e-ac9c-4386-acac-2106d84bebc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:13.150861 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a7c09e-ac9c-4386-acac-2106d84bebc6 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:13.161877 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1406/2827] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:23:13.166839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f926c-515a-4d52-b416-cc503ad75540 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:13.176974 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f926c-515a-4d52-b416-cc503ad75540 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:13.182486 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd146535-f4d0-4db6-a514-86a29d30cdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:13.183174 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd146535-f4d0-4db6-a514-86a29d30cdf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:13.183488 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd146535-f4d0-4db6-a514-86a29d30cdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:13.183769 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd146535-f4d0-4db6-a514-86a29d30cdf3 admin admin] PATH_INFO: `/v3/projects/ce7091591a71472b9204314cdf4c3b57/users/4b38f7b1b310437abcfe4bc77436f7d3/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:13.198841 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd146535-f4d0-4db6-a514-86a29d30cdf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce7091591a71472b9204314cdf4c3b57, user_id=4b38f7b1b310437abcfe4bc77436f7d3, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:13.201582 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd146535-f4d0-4db6-a514-86a29d30cdf3 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:13.258011 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1422/2828] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:23:13 2022] PUT /identity/v3/projects/ce7091591a71472b9204314cdf4c3b57/users/4b38f7b1b310437abcfe4bc77436f7d3/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:13.261938 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7c09e-ac9c-4386-acac-2106d84bebc6 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:13.272010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a7c09e-ac9c-4386-acac-2106d84bebc6 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:13.312819 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:13.328903 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5a47eb-430d-4b57-bfaa-ebfab16ee458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:13.329588 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a47eb-430d-4b57-bfaa-ebfab16ee458 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:13.329874 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a47eb-430d-4b57-bfaa-ebfab16ee458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:13.330253 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a47eb-430d-4b57-bfaa-ebfab16ee458 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:13.330948 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a47eb-430d-4b57-bfaa-ebfab16ee458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:13.332534 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5a47eb-430d-4b57-bfaa-ebfab16ee458 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:13.342617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1407/2829] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:23:13.346866 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd146535-f4d0-4db6-a514-86a29d30cdf3 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:13.357638 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd146535-f4d0-4db6-a514-86a29d30cdf3 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:13.362346 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378fd261-820d-4081-9b6e-6263a09a9d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:13.362997 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fd261-820d-4081-9b6e-6263a09a9d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:13.367550 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fd261-820d-4081-9b6e-6263a09a9d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:13.367550 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378fd261-820d-4081-9b6e-6263a09a9d16 admin admin] PATH_INFO: `/v3/projects/ce7091591a71472b9204314cdf4c3b57/users/4b38f7b1b310437abcfe4bc77436f7d3/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:13.367550 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378fd261-820d-4081-9b6e-6263a09a9d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce7091591a71472b9204314cdf4c3b57, user_id=4b38f7b1b310437abcfe4bc77436f7d3, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:13.368104 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378fd261-820d-4081-9b6e-6263a09a9d16 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:13.378011 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1423/2830] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:23:13 2022] PUT /identity/v3/projects/ce7091591a71472b9204314cdf4c3b57/users/4b38f7b1b310437abcfe4bc77436f7d3/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:13.390498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c0523-584c-4c56-8ac0-09047b626f4b None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:13.390498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c0523-584c-4c56-8ac0-09047b626f4b None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:13.390498 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1c0523-584c-4c56-8ac0-09047b626f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:13.420409 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-ed1c0523-584c-4c56-8ac0-09047b626f4b None None] MFA Rules not processed for user `4b38f7b1b310437abcfe4bc77436f7d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:23:13.421621 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-ed1c0523-584c-4c56-8ac0-09047b626f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:13.489810 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1408/2831] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:23:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:23:13.501631 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378fd261-820d-4081-9b6e-6263a09a9d16 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:13.503329 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-378fd261-820d-4081-9b6e-6263a09a9d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:13.536136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:13.582057 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378fd261-820d-4081-9b6e-6263a09a9d16 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:13.585662 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865bb40-c11f-4cea-88fc-1129ff6e4142 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:13.586360 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865bb40-c11f-4cea-88fc-1129ff6e4142 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:13.586642 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865bb40-c11f-4cea-88fc-1129ff6e4142 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:13.586964 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f865bb40-c11f-4cea-88fc-1129ff6e4142 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:13.596168 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865bb40-c11f-4cea-88fc-1129ff6e4142 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:13.597596 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f865bb40-c11f-4cea-88fc-1129ff6e4142 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:13.607765 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1424/2832] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:23:13 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:23:13.820068 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c0523-584c-4c56-8ac0-09047b626f4b None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:13.829712 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1c0523-584c-4c56-8ac0-09047b626f4b None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:13.833431 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd557a7b-bd8b-464b-b617-4aec9cb455c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:13.834081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd557a7b-bd8b-464b-b617-4aec9cb455c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:13.834363 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd557a7b-bd8b-464b-b617-4aec9cb455c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:13.834636 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd557a7b-bd8b-464b-b617-4aec9cb455c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:13.890539 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd557a7b-bd8b-464b-b617-4aec9cb455c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:13.892000 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd557a7b-bd8b-464b-b617-4aec9cb455c8 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:13.901528 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1409/2833] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:23:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:23:17.238258 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865bb40-c11f-4cea-88fc-1129ff6e4142 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:17.299735 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f865bb40-c11f-4cea-88fc-1129ff6e4142 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:17.304424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7edc3e-8213-42a2-9aa2-c6e29144ca9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:17.305057 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7edc3e-8213-42a2-9aa2-c6e29144ca9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:17.305335 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7edc3e-8213-42a2-9aa2-c6e29144ca9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:17.305607 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7edc3e-8213-42a2-9aa2-c6e29144ca9b admin admin] PATH_INFO: `/v3/users/0beb5f1b5ef949c4a7cbe9361a10a44a` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:17.306790 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7edc3e-8213-42a2-9aa2-c6e29144ca9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0beb5f1b5ef949c4a7cbe9361a10a44a)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:17.308679 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7edc3e-8213-42a2-9aa2-c6e29144ca9b admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:17.487134 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0c7edc3e-8213-42a2-9aa2-c6e29144ca9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0beb5f1b5ef949c4a7cbe9361a10a44a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:17.494138 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0c7edc3e-8213-42a2-9aa2-c6e29144ca9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0beb5f1b5ef949c4a7cbe9361a10a44a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:17.494792 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0c7edc3e-8213-42a2-9aa2-c6e29144ca9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0beb5f1b5ef949c4a7cbe9361a10a44a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:17.506682 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0c7edc3e-8213-42a2-9aa2-c6e29144ca9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0beb5f1b5ef949c4a7cbe9361a10a44a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:17.507336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0c7edc3e-8213-42a2-9aa2-c6e29144ca9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0beb5f1b5ef949c4a7cbe9361a10a44a'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:17.571427 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1425/2834] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:23:17 2022] DELETE /identity/v3/users/0beb5f1b5ef949c4a7cbe9361a10a44a => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:17.575797 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05c93b-d542-4835-9c8c-46595b7a50be None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:17.576318 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05c93b-d542-4835-9c8c-46595b7a50be None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:17.576433 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05c93b-d542-4835-9c8c-46595b7a50be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:17.598629 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-9f05c93b-d542-4835-9c8c-46595b7a50be None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:23:17.599846 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-9f05c93b-d542-4835-9c8c-46595b7a50be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:17.656873 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1410/2835] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:23:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:23:17.666713 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7edc3e-8213-42a2-9aa2-c6e29144ca9b admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:17.668130 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0c7edc3e-8213-42a2-9aa2-c6e29144ca9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:17.703480 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:17.724039 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7edc3e-8213-42a2-9aa2-c6e29144ca9b admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:17.732801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e9144-d8c8-47ee-8503-889857a8e343 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:17.732801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e9144-d8c8-47ee-8503-889857a8e343 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:17.732801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e9144-d8c8-47ee-8503-889857a8e343 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:17.732801 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e9144-d8c8-47ee-8503-889857a8e343 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:17.737755 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e9144-d8c8-47ee-8503-889857a8e343 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:17.739215 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e9144-d8c8-47ee-8503-889857a8e343 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:17.749630 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1426/2836] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:23:18.105151 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05c93b-d542-4835-9c8c-46595b7a50be None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:18.106609 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-9f05c93b-d542-4835-9c8c-46595b7a50be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:18.114163 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:18.120266 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05c93b-d542-4835-9c8c-46595b7a50be None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:18.125170 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb00cd3-f53e-4846-9003-9ed3ef920e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:18.125830 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb00cd3-f53e-4846-9003-9ed3ef920e08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:18.126145 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb00cd3-f53e-4846-9003-9ed3ef920e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:18.138023 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb00cd3-f53e-4846-9003-9ed3ef920e08 admin admin] PATH_INFO: `/v3/projects/79a7356630c34973a52df71de19dbc0a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:18.138023 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb00cd3-f53e-4846-9003-9ed3ef920e08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79a7356630c34973a52df71de19dbc0a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:18.138023 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb00cd3-f53e-4846-9003-9ed3ef920e08 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:18.235794 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-0cb00cd3-f53e-4846-9003-9ed3ef920e08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79a7356630c34973a52df71de19dbc0a', 'request_id': 'req-0cb00cd3-f53e-4846-9003-9ed3ef920e08'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:18.242864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-0cb00cd3-f53e-4846-9003-9ed3ef920e08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79a7356630c34973a52df71de19dbc0a', 'request_id': 'req-0cb00cd3-f53e-4846-9003-9ed3ef920e08'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:18.248395 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-0cb00cd3-f53e-4846-9003-9ed3ef920e08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79a7356630c34973a52df71de19dbc0a', 'request_id': 'req-0cb00cd3-f53e-4846-9003-9ed3ef920e08'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:18.302503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-0cb00cd3-f53e-4846-9003-9ed3ef920e08 admin admin] The token cache is being invalidate because project 79a7356630c34973a52df71de19dbc0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:23:18.302788 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-0cb00cd3-f53e-4846-9003-9ed3ef920e08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:18.303651 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-0cb00cd3-f53e-4846-9003-9ed3ef920e08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:18.305737 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1411/2837] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:23:18 2022] DELETE /identity/v3/projects/79a7356630c34973a52df71de19dbc0a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:18.310025 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e9144-d8c8-47ee-8503-889857a8e343 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:18.320030 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e9144-d8c8-47ee-8503-889857a8e343 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:18.376071 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe45e7ff-d3d0-4444-ab96-34f52fe354d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:18.376628 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45e7ff-d3d0-4444-ab96-34f52fe354d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:18.376914 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45e7ff-d3d0-4444-ab96-34f52fe354d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:18.377194 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45e7ff-d3d0-4444-ab96-34f52fe354d2 admin admin] PATH_INFO: `/v3/users/4b38f7b1b310437abcfe4bc77436f7d3` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:18.378384 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45e7ff-d3d0-4444-ab96-34f52fe354d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b38f7b1b310437abcfe4bc77436f7d3)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:18.380309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe45e7ff-d3d0-4444-ab96-34f52fe354d2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:18.510367 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fe45e7ff-d3d0-4444-ab96-34f52fe354d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b38f7b1b310437abcfe4bc77436f7d3'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:18.517131 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fe45e7ff-d3d0-4444-ab96-34f52fe354d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b38f7b1b310437abcfe4bc77436f7d3'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:18.517784 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fe45e7ff-d3d0-4444-ab96-34f52fe354d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b38f7b1b310437abcfe4bc77436f7d3'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:18.529849 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fe45e7ff-d3d0-4444-ab96-34f52fe354d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b38f7b1b310437abcfe4bc77436f7d3'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:18.530505 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-fe45e7ff-d3d0-4444-ab96-34f52fe354d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b38f7b1b310437abcfe4bc77436f7d3'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:18.593951 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1427/2838] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:23:18 2022] DELETE /identity/v3/users/4b38f7b1b310437abcfe4bc77436f7d3 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:18.973465 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb00cd3-f53e-4846-9003-9ed3ef920e08 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:18.974928 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-0cb00cd3-f53e-4846-9003-9ed3ef920e08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:18.987844 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb00cd3-f53e-4846-9003-9ed3ef920e08 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:19.047516 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae7652e-82ae-4233-b60b-285f1140301a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:19.048164 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7652e-82ae-4233-b60b-285f1140301a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:19.048452 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7652e-82ae-4233-b60b-285f1140301a admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:19.048853 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae7652e-82ae-4233-b60b-285f1140301a admin admin] PATH_INFO: `/v3/projects/ce7091591a71472b9204314cdf4c3b57` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:19.050084 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae7652e-82ae-4233-b60b-285f1140301a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce7091591a71472b9204314cdf4c3b57)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:19.052018 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae7652e-82ae-4233-b60b-285f1140301a admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:19.132261 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-dae7652e-82ae-4233-b60b-285f1140301a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce7091591a71472b9204314cdf4c3b57', 'request_id': 'req-dae7652e-82ae-4233-b60b-285f1140301a'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:19.139385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-dae7652e-82ae-4233-b60b-285f1140301a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce7091591a71472b9204314cdf4c3b57', 'request_id': 'req-dae7652e-82ae-4233-b60b-285f1140301a'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:19.144999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-dae7652e-82ae-4233-b60b-285f1140301a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce7091591a71472b9204314cdf4c3b57', 'request_id': 'req-dae7652e-82ae-4233-b60b-285f1140301a'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:19.188024 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-dae7652e-82ae-4233-b60b-285f1140301a admin admin] The token cache is being invalidate because project ce7091591a71472b9204314cdf4c3b57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:23:19.188310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-dae7652e-82ae-4233-b60b-285f1140301a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:19.188917 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-dae7652e-82ae-4233-b60b-285f1140301a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:19.190837 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1412/2839] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:23:18 2022] DELETE /identity/v3/projects/ce7091591a71472b9204314cdf4c3b57 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:19.196022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9d200-5bae-48b5-a820-0710e883f69d None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:19.196310 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9d200-5bae-48b5-a820-0710e883f69d None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:19.196708 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9d200-5bae-48b5-a820-0710e883f69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:19.217031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f5d9d200-5bae-48b5-a820-0710e883f69d None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:23:19.218252 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f5d9d200-5bae-48b5-a820-0710e883f69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:19.264686 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:19.275808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1428/2840] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:23:19.283570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486217f7-e752-4e11-95bb-db3ebf2fc0e9 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:19.283858 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486217f7-e752-4e11-95bb-db3ebf2fc0e9 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:19.284134 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486217f7-e752-4e11-95bb-db3ebf2fc0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:19.303769 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-486217f7-e752-4e11-95bb-db3ebf2fc0e9 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:23:19.305559 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-486217f7-e752-4e11-95bb-db3ebf2fc0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:19.313635 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1413/2841] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:23:19.317754 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9d200-5bae-48b5-a820-0710e883f69d None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:19.328282 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9d200-5bae-48b5-a820-0710e883f69d None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:19.334237 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439cb55-6685-4dbe-979c-c54ff0b195db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:19.334237 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439cb55-6685-4dbe-979c-c54ff0b195db admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:19.334237 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439cb55-6685-4dbe-979c-c54ff0b195db admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:19.334237 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0439cb55-6685-4dbe-979c-c54ff0b195db admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:19.334952 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439cb55-6685-4dbe-979c-c54ff0b195db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:23:19.335256 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439cb55-6685-4dbe-979c-c54ff0b195db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:19.337347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0439cb55-6685-4dbe-979c-c54ff0b195db admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:19.349022 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1429/2842] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:23:19.353073 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486217f7-e752-4e11-95bb-db3ebf2fc0e9 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:19.362785 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486217f7-e752-4e11-95bb-db3ebf2fc0e9 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:19.368050 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f05183-bd4a-40ff-b8ec-a7dd84648c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:19.368658 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f05183-bd4a-40ff-b8ec-a7dd84648c8d admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:19.368941 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f05183-bd4a-40ff-b8ec-a7dd84648c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:19.369214 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f05183-bd4a-40ff-b8ec-a7dd84648c8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:19.369916 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f05183-bd4a-40ff-b8ec-a7dd84648c8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:19.371820 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f05183-bd4a-40ff-b8ec-a7dd84648c8d admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:19.377397 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:19.475524 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1414/2843] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:23:19 2022] POST /identity/v3/projects => generated 407 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:23:19.480169 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439cb55-6685-4dbe-979c-c54ff0b195db admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:19.490105 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0439cb55-6685-4dbe-979c-c54ff0b195db admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:19.547178 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2da91-1d60-48c3-9e69-9e215414026f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:19.547821 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2da91-1d60-48c3-9e69-9e215414026f admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:19.548100 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2da91-1d60-48c3-9e69-9e215414026f admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:19.548376 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd2da91-1d60-48c3-9e69-9e215414026f admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:19.549056 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2da91-1d60-48c3-9e69-9e215414026f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:19.550988 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd2da91-1d60-48c3-9e69-9e215414026f admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:19.625345 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1430/2844] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:23:19 2022] POST /identity/v3/users => generated 304 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:23:19.629589 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f05183-bd4a-40ff-b8ec-a7dd84648c8d admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:19.640175 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f05183-bd4a-40ff-b8ec-a7dd84648c8d admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:19.644895 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721cf70d-29ef-46db-bdeb-f01917bac748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:19.645503 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cf70d-29ef-46db-bdeb-f01917bac748 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:19.645787 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cf70d-29ef-46db-bdeb-f01917bac748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:19.646103 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cf70d-29ef-46db-bdeb-f01917bac748 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:19.646963 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721cf70d-29ef-46db-bdeb-f01917bac748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:19.648562 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721cf70d-29ef-46db-bdeb-f01917bac748 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:19.658925 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1415/2845] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:23:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:23:19.662944 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2da91-1d60-48c3-9e69-9e215414026f admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:19.673446 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd2da91-1d60-48c3-9e69-9e215414026f admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:19.678417 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30310a07-5ed6-45fc-baf1-1d08374c0414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:19.679018 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30310a07-5ed6-45fc-baf1-1d08374c0414 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:19.679298 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30310a07-5ed6-45fc-baf1-1d08374c0414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:19.680324 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30310a07-5ed6-45fc-baf1-1d08374c0414 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:19.681069 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30310a07-5ed6-45fc-baf1-1d08374c0414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:19.682833 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30310a07-5ed6-45fc-baf1-1d08374c0414 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:19.693709 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1431/2846] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:23:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 09:23:19.698078 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721cf70d-29ef-46db-bdeb-f01917bac748 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:19.708519 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721cf70d-29ef-46db-bdeb-f01917bac748 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:19.714083 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1ac85-f92a-4fd6-94a7-589fcc9fbb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:19.714754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1ac85-f92a-4fd6-94a7-589fcc9fbb33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:19.715236 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1ac85-f92a-4fd6-94a7-589fcc9fbb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:19.715732 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1ac85-f92a-4fd6-94a7-589fcc9fbb33 admin admin] PATH_INFO: `/v3/projects/3fefcf78f0424c17ac9a5aefb463fd09/users/8f40a37094974ea198dc5cc60f2323dc/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:19.733144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba1ac85-f92a-4fd6-94a7-589fcc9fbb33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fefcf78f0424c17ac9a5aefb463fd09, user_id=8f40a37094974ea198dc5cc60f2323dc, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:19.735760 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba1ac85-f92a-4fd6-94a7-589fcc9fbb33 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:19.840063 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1416/2847] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:23:19 2022] PUT /identity/v3/projects/3fefcf78f0424c17ac9a5aefb463fd09/users/8f40a37094974ea198dc5cc60f2323dc/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:19.910287 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb8c01-b731-49c4-8a90-c535c0a2c27a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:19.910579 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb8c01-b731-49c4-8a90-c535c0a2c27a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:19.910861 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb8c01-b731-49c4-8a90-c535c0a2c27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:19.939128 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-4dcb8c01-b731-49c4-8a90-c535c0a2c27a None None] MFA Rules not processed for user `8f40a37094974ea198dc5cc60f2323dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:23:19.940347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4dcb8c01-b731-49c4-8a90-c535c0a2c27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:20.002745 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1432/2848] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:23:19 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:23:20.007254 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1ac85-f92a-4fd6-94a7-589fcc9fbb33 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:20.068069 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1ac85-f92a-4fd6-94a7-589fcc9fbb33 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:20.072666 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611be6d-fec1-43b3-ae71-8ad5183dd294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:20.073270 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611be6d-fec1-43b3-ae71-8ad5183dd294 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:20.073546 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611be6d-fec1-43b3-ae71-8ad5183dd294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:20.073816 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611be6d-fec1-43b3-ae71-8ad5183dd294 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:20.074499 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0611be6d-fec1-43b3-ae71-8ad5183dd294 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:20.076381 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0611be6d-fec1-43b3-ae71-8ad5183dd294 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:20.150081 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1417/2849] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:23:20 2022] POST /identity/v3/projects => generated 407 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:23:20.154463 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb8c01-b731-49c4-8a90-c535c0a2c27a None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:20.165696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb8c01-b731-49c4-8a90-c535c0a2c27a None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:20.212726 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:20.222188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c34229-daa7-4783-8807-cdf16b43d35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:20.222808 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c34229-daa7-4783-8807-cdf16b43d35d admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:20.223093 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c34229-daa7-4783-8807-cdf16b43d35d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:20.227061 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c34229-daa7-4783-8807-cdf16b43d35d admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:20.227061 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c34229-daa7-4783-8807-cdf16b43d35d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:20.227061 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c34229-daa7-4783-8807-cdf16b43d35d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:20.297233 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1433/2850] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:23:20 2022] POST /identity/v3/users => generated 310 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:23:20.300837 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611be6d-fec1-43b3-ae71-8ad5183dd294 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:20.311502 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0611be6d-fec1-43b3-ae71-8ad5183dd294 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:20.316455 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc15f19d-8a03-42a3-9a9b-517a7f5137b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:20.317069 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15f19d-8a03-42a3-9a9b-517a7f5137b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:20.317356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15f19d-8a03-42a3-9a9b-517a7f5137b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:20.317639 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc15f19d-8a03-42a3-9a9b-517a7f5137b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:20.318381 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc15f19d-8a03-42a3-9a9b-517a7f5137b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:20.320620 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc15f19d-8a03-42a3-9a9b-517a7f5137b5 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:20.331576 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1418/2851] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:23:20.336299 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c34229-daa7-4783-8807-cdf16b43d35d admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:20.346494 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c34229-daa7-4783-8807-cdf16b43d35d admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:20.351240 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d139ef-3212-4133-a8ce-50300e40845c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:20.351893 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d139ef-3212-4133-a8ce-50300e40845c admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:20.352175 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d139ef-3212-4133-a8ce-50300e40845c admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:20.352451 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d139ef-3212-4133-a8ce-50300e40845c admin admin] PATH_INFO: `/v3/projects/b56d21d3e3294b18afcf0e4128af4b3c/users/351788d909d7408995451b01af8672ad/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:20.366743 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d139ef-3212-4133-a8ce-50300e40845c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b56d21d3e3294b18afcf0e4128af4b3c, user_id=351788d909d7408995451b01af8672ad, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:20.369383 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d139ef-3212-4133-a8ce-50300e40845c admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:20.428947 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1434/2852] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:23:20 2022] PUT /identity/v3/projects/b56d21d3e3294b18afcf0e4128af4b3c/users/351788d909d7408995451b01af8672ad/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:20.432997 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc15f19d-8a03-42a3-9a9b-517a7f5137b5 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:20.443035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc15f19d-8a03-42a3-9a9b-517a7f5137b5 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:20.498107 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a2402-9828-4172-9f67-ee25fa3ddc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:20.498725 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a2402-9828-4172-9f67-ee25fa3ddc62 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:20.499010 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a2402-9828-4172-9f67-ee25fa3ddc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:20.499290 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a2402-9828-4172-9f67-ee25fa3ddc62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:20.500014 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a2402-9828-4172-9f67-ee25fa3ddc62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:20.501601 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a2402-9828-4172-9f67-ee25fa3ddc62 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:20.511700 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1419/2853] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:23:20 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:23:20.515818 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d139ef-3212-4133-a8ce-50300e40845c admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:20.526429 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d139ef-3212-4133-a8ce-50300e40845c admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:20.531031 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e9b47-c1c9-40d4-b09a-b684ba19d070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:20.531675 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e9b47-c1c9-40d4-b09a-b684ba19d070 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:20.531953 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e9b47-c1c9-40d4-b09a-b684ba19d070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:20.532227 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e9b47-c1c9-40d4-b09a-b684ba19d070 admin admin] PATH_INFO: `/v3/projects/b56d21d3e3294b18afcf0e4128af4b3c/users/351788d909d7408995451b01af8672ad/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:20.534183 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e9b47-c1c9-40d4-b09a-b684ba19d070 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b56d21d3e3294b18afcf0e4128af4b3c, user_id=351788d909d7408995451b01af8672ad, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:20.536783 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e9b47-c1c9-40d4-b09a-b684ba19d070 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:20.546793 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1435/2854] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:23:20 2022] PUT /identity/v3/projects/b56d21d3e3294b18afcf0e4128af4b3c/users/351788d909d7408995451b01af8672ad/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:20.556426 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621f2a6-6834-41af-8f05-e19b88a8184d None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:20.556658 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621f2a6-6834-41af-8f05-e19b88a8184d None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:20.556940 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b621f2a6-6834-41af-8f05-e19b88a8184d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:20.584318 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-b621f2a6-6834-41af-8f05-e19b88a8184d None None] MFA Rules not processed for user `351788d909d7408995451b01af8672ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:23:20.585515 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-b621f2a6-6834-41af-8f05-e19b88a8184d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:20.595015 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:20.652686 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1420/2855] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:23:20 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:23:20.663743 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e9b47-c1c9-40d4-b09a-b684ba19d070 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:20.665522 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-7b8e9b47-c1c9-40d4-b09a-b684ba19d070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:20.723466 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e9b47-c1c9-40d4-b09a-b684ba19d070 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:20.727450 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93c2e0-590f-4bc0-837c-d50ccf0b48b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:20.728063 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93c2e0-590f-4bc0-837c-d50ccf0b48b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:20.728347 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93c2e0-590f-4bc0-837c-d50ccf0b48b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:20.728618 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93c2e0-590f-4bc0-837c-d50ccf0b48b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:20.737482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf93c2e0-590f-4bc0-837c-d50ccf0b48b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:20.738939 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf93c2e0-590f-4bc0-837c-d50ccf0b48b5 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:20.748952 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1436/2856] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:23:20 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:23:21.072985 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621f2a6-6834-41af-8f05-e19b88a8184d None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:21.082789 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b621f2a6-6834-41af-8f05-e19b88a8184d None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:21.086398 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783d9ac-07d9-458a-916c-3efa7cadf630 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:21.087009 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783d9ac-07d9-458a-916c-3efa7cadf630 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:21.087297 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783d9ac-07d9-458a-916c-3efa7cadf630 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:21.087575 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783d9ac-07d9-458a-916c-3efa7cadf630 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:21.144275 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783d9ac-07d9-458a-916c-3efa7cadf630 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:21.145749 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783d9ac-07d9-458a-916c-3efa7cadf630 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:21.155221 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1421/2857] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:23:21 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:23:30.744677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=47,keystone:INSERT=5 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:30.842858 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93c2e0-590f-4bc0-837c-d50ccf0b48b5 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:30.903579 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93c2e0-590f-4bc0-837c-d50ccf0b48b5 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:30.908090 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233102c-302e-4e96-89da-55d9ed3260c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:30.908721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233102c-302e-4e96-89da-55d9ed3260c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:30.908998 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233102c-302e-4e96-89da-55d9ed3260c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:30.909271 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233102c-302e-4e96-89da-55d9ed3260c2 admin admin] PATH_INFO: `/v3/users/8f40a37094974ea198dc5cc60f2323dc` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:30.910467 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233102c-302e-4e96-89da-55d9ed3260c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f40a37094974ea198dc5cc60f2323dc)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:30.912361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233102c-302e-4e96-89da-55d9ed3260c2 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:31.070605 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0233102c-302e-4e96-89da-55d9ed3260c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f40a37094974ea198dc5cc60f2323dc'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:31.077994 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0233102c-302e-4e96-89da-55d9ed3260c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f40a37094974ea198dc5cc60f2323dc'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:31.078734 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0233102c-302e-4e96-89da-55d9ed3260c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f40a37094974ea198dc5cc60f2323dc'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:31.091290 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0233102c-302e-4e96-89da-55d9ed3260c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f40a37094974ea198dc5cc60f2323dc'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:31.091927 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-0233102c-302e-4e96-89da-55d9ed3260c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f40a37094974ea198dc5cc60f2323dc'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:31.150937 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=52 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:31.194696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1437/2858] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:23:30 2022] DELETE /identity/v3/users/8f40a37094974ea198dc5cc60f2323dc => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:31.200327 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5599d-acc4-4990-b3b8-bedf45ecdb8b None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:31.200468 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5599d-acc4-4990-b3b8-bedf45ecdb8b None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:31.200656 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5599d-acc4-4990-b3b8-bedf45ecdb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:31.221354 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-e9f5599d-acc4-4990-b3b8-bedf45ecdb8b None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:23:31.222587 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e9f5599d-acc4-4990-b3b8-bedf45ecdb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:31.279522 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1422/2859] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:23:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:23:31.289121 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233102c-302e-4e96-89da-55d9ed3260c2 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:31.290619 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-0233102c-302e-4e96-89da-55d9ed3260c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:31.348707 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233102c-302e-4e96-89da-55d9ed3260c2 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:31.352295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b04d22-dbe3-475d-a2c9-bbc8d81dbcc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:31.352968 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b04d22-dbe3-475d-a2c9-bbc8d81dbcc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:31.353250 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b04d22-dbe3-475d-a2c9-bbc8d81dbcc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:31.353528 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b04d22-dbe3-475d-a2c9-bbc8d81dbcc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:31.362476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b04d22-dbe3-475d-a2c9-bbc8d81dbcc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:31.363896 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b04d22-dbe3-475d-a2c9-bbc8d81dbcc6 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:31.373796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1438/2860] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:23:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:23:31.744392 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5599d-acc4-4990-b3b8-bedf45ecdb8b None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:31.745851 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e9f5599d-acc4-4990-b3b8-bedf45ecdb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:31.758045 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5599d-acc4-4990-b3b8-bedf45ecdb8b None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:31.763634 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5405a5-2df5-424d-8a8f-419ced28b87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:31.765032 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5405a5-2df5-424d-8a8f-419ced28b87d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:31.765315 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5405a5-2df5-424d-8a8f-419ced28b87d admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:31.765748 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5405a5-2df5-424d-8a8f-419ced28b87d admin admin] PATH_INFO: `/v3/projects/3fefcf78f0424c17ac9a5aefb463fd09` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:31.766969 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5405a5-2df5-424d-8a8f-419ced28b87d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fefcf78f0424c17ac9a5aefb463fd09)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:31.768873 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5405a5-2df5-424d-8a8f-419ced28b87d admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:31.897657 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-1e5405a5-2df5-424d-8a8f-419ced28b87d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fefcf78f0424c17ac9a5aefb463fd09', 'request_id': 'req-1e5405a5-2df5-424d-8a8f-419ced28b87d'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:31.904136 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-1e5405a5-2df5-424d-8a8f-419ced28b87d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fefcf78f0424c17ac9a5aefb463fd09', 'request_id': 'req-1e5405a5-2df5-424d-8a8f-419ced28b87d'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:31.909574 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-1e5405a5-2df5-424d-8a8f-419ced28b87d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fefcf78f0424c17ac9a5aefb463fd09', 'request_id': 'req-1e5405a5-2df5-424d-8a8f-419ced28b87d'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:31.987880 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-1e5405a5-2df5-424d-8a8f-419ced28b87d admin admin] The token cache is being invalidate because project 3fefcf78f0424c17ac9a5aefb463fd09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:23:31.988176 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-1e5405a5-2df5-424d-8a8f-419ced28b87d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:31.988797 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-1e5405a5-2df5-424d-8a8f-419ced28b87d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:31.990381 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1423/2861] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:23:31 2022] DELETE /identity/v3/projects/3fefcf78f0424c17ac9a5aefb463fd09 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:31.994888 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b04d22-dbe3-475d-a2c9-bbc8d81dbcc6 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:32.004640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b04d22-dbe3-475d-a2c9-bbc8d81dbcc6 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:32.022875 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:32.060460 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d32d8-732b-47f3-ad2f-15418e3967b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:32.061116 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d32d8-732b-47f3-ad2f-15418e3967b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:32.061424 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d32d8-732b-47f3-ad2f-15418e3967b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:32.061704 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d32d8-732b-47f3-ad2f-15418e3967b3 admin admin] PATH_INFO: `/v3/users/351788d909d7408995451b01af8672ad` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:32.062911 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d32d8-732b-47f3-ad2f-15418e3967b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=351788d909d7408995451b01af8672ad)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:32.064863 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d32d8-732b-47f3-ad2f-15418e3967b3 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:32.299549 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9a3d32d8-732b-47f3-ad2f-15418e3967b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '351788d909d7408995451b01af8672ad'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:32.307302 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9a3d32d8-732b-47f3-ad2f-15418e3967b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '351788d909d7408995451b01af8672ad'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:32.307985 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9a3d32d8-732b-47f3-ad2f-15418e3967b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '351788d909d7408995451b01af8672ad'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:32.319787 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9a3d32d8-732b-47f3-ad2f-15418e3967b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '351788d909d7408995451b01af8672ad'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:32.320420 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-9a3d32d8-732b-47f3-ad2f-15418e3967b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '351788d909d7408995451b01af8672ad'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:32.394141 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1439/2862] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:23:31 2022] DELETE /identity/v3/users/351788d909d7408995451b01af8672ad => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:32.784111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5405a5-2df5-424d-8a8f-419ced28b87d admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:32.785630 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-1e5405a5-2df5-424d-8a8f-419ced28b87d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:32.800304 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5405a5-2df5-424d-8a8f-419ced28b87d admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:32.857710 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236390a1-d87a-4335-bdd1-b48993e43190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:32.858383 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236390a1-d87a-4335-bdd1-b48993e43190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:32.858670 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236390a1-d87a-4335-bdd1-b48993e43190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:32.858951 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236390a1-d87a-4335-bdd1-b48993e43190 admin admin] PATH_INFO: `/v3/projects/b56d21d3e3294b18afcf0e4128af4b3c` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:32.860106 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236390a1-d87a-4335-bdd1-b48993e43190 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b56d21d3e3294b18afcf0e4128af4b3c)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:32.862039 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236390a1-d87a-4335-bdd1-b48993e43190 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:32.938806 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:32.955152 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-236390a1-d87a-4335-bdd1-b48993e43190 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b56d21d3e3294b18afcf0e4128af4b3c', 'request_id': 'req-236390a1-d87a-4335-bdd1-b48993e43190'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:32.961471 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-236390a1-d87a-4335-bdd1-b48993e43190 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b56d21d3e3294b18afcf0e4128af4b3c', 'request_id': 'req-236390a1-d87a-4335-bdd1-b48993e43190'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:32.967864 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-236390a1-d87a-4335-bdd1-b48993e43190 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b56d21d3e3294b18afcf0e4128af4b3c', 'request_id': 'req-236390a1-d87a-4335-bdd1-b48993e43190'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:33.021673 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-236390a1-d87a-4335-bdd1-b48993e43190 admin admin] The token cache is being invalidate because project b56d21d3e3294b18afcf0e4128af4b3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:23:33.022205 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-236390a1-d87a-4335-bdd1-b48993e43190 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:33.022955 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-236390a1-d87a-4335-bdd1-b48993e43190 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:33.025485 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1424/2863] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:23:32 2022] DELETE /identity/v3/projects/b56d21d3e3294b18afcf0e4128af4b3c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:33.030616 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f4c71-a9bb-4fec-893d-d9914c143ba7 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:33.031109 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f4c71-a9bb-4fec-893d-d9914c143ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:33.031225 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f4c71-a9bb-4fec-893d-d9914c143ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:33.050120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-435f4c71-a9bb-4fec-893d-d9914c143ba7 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:23:33.052355 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-435f4c71-a9bb-4fec-893d-d9914c143ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:33.109766 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1440/2864] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:23:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:23:33.118512 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ab6af-489c-4a5e-90d0-1b6c45a667c1 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:33.118806 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ab6af-489c-4a5e-90d0-1b6c45a667c1 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:33.119084 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1ab6af-489c-4a5e-90d0-1b6c45a667c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:33.138970 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-2f1ab6af-489c-4a5e-90d0-1b6c45a667c1 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:23:33.140179 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-2f1ab6af-489c-4a5e-90d0-1b6c45a667c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:33.148172 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1425/2865] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:23:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:23:33.152890 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f4c71-a9bb-4fec-893d-d9914c143ba7 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:33.162435 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f4c71-a9bb-4fec-893d-d9914c143ba7 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:33.167035 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ba5a8-ff33-49a1-8b20-d6d624650763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:33.167648 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ba5a8-ff33-49a1-8b20-d6d624650763 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:33.167931 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ba5a8-ff33-49a1-8b20-d6d624650763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:33.168208 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1ba5a8-ff33-49a1-8b20-d6d624650763 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:33.168894 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ba5a8-ff33-49a1-8b20-d6d624650763 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:23:33.169199 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ba5a8-ff33-49a1-8b20-d6d624650763 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:33.170816 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1ba5a8-ff33-49a1-8b20-d6d624650763 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:33.182418 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1441/2866] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:23:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:23:33.186749 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ab6af-489c-4a5e-90d0-1b6c45a667c1 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:33.196139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1ab6af-489c-4a5e-90d0-1b6c45a667c1 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:33.201461 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf18c3-b4a2-4d15-85bc-04b439db2cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:33.202096 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf18c3-b4a2-4d15-85bc-04b439db2cfd admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:33.202405 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf18c3-b4a2-4d15-85bc-04b439db2cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:33.202680 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf18c3-b4a2-4d15-85bc-04b439db2cfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:33.203352 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf18c3-b4a2-4d15-85bc-04b439db2cfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:33.206197 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdf18c3-b4a2-4d15-85bc-04b439db2cfd admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:33.320917 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1426/2867] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:23:33 2022] POST /identity/v3/projects => generated 367 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:23:33.325805 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ba5a8-ff33-49a1-8b20-d6d624650763 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:33.335830 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1ba5a8-ff33-49a1-8b20-d6d624650763 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:33.360995 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:33.391411 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5548a12-059f-49c6-88ea-0e3c7916c3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:33.392015 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5548a12-059f-49c6-88ea-0e3c7916c3eb admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:33.392292 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5548a12-059f-49c6-88ea-0e3c7916c3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:33.392563 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5548a12-059f-49c6-88ea-0e3c7916c3eb admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:33.393218 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5548a12-059f-49c6-88ea-0e3c7916c3eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:33.395140 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5548a12-059f-49c6-88ea-0e3c7916c3eb admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:33.497136 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1442/2868] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:23:33 2022] POST /identity/v3/users => generated 284 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:23:33.501095 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf18c3-b4a2-4d15-85bc-04b439db2cfd admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:33.511965 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf18c3-b4a2-4d15-85bc-04b439db2cfd admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:33.516629 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93686407-c988-4020-9fc4-ade4d550e8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:33.517247 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93686407-c988-4020-9fc4-ade4d550e8f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:33.517526 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93686407-c988-4020-9fc4-ade4d550e8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:33.517799 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93686407-c988-4020-9fc4-ade4d550e8f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:33.518572 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93686407-c988-4020-9fc4-ade4d550e8f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:33.520156 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93686407-c988-4020-9fc4-ade4d550e8f1 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:33.529946 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1427/2869] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:23:33.534033 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5548a12-059f-49c6-88ea-0e3c7916c3eb admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:33.543949 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5548a12-059f-49c6-88ea-0e3c7916c3eb admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:33.548516 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fefb3d-21d8-49cb-be0e-a830eae3fdd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:33.549120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fefb3d-21d8-49cb-be0e-a830eae3fdd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:33.549404 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fefb3d-21d8-49cb-be0e-a830eae3fdd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:33.549679 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fefb3d-21d8-49cb-be0e-a830eae3fdd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:33.550410 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fefb3d-21d8-49cb-be0e-a830eae3fdd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:33.552013 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fefb3d-21d8-49cb-be0e-a830eae3fdd7 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:33.563086 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1443/2870] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:23:33.567047 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93686407-c988-4020-9fc4-ade4d550e8f1 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:33.577584 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93686407-c988-4020-9fc4-ade4d550e8f1 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:33.582281 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff7958-cde3-4ce9-9850-d336845fc61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:33.582933 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff7958-cde3-4ce9-9850-d336845fc61e admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:33.583218 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff7958-cde3-4ce9-9850-d336845fc61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:33.583496 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff7958-cde3-4ce9-9850-d336845fc61e admin admin] PATH_INFO: `/v3/projects/5e26332398f34ada9b7d568f01c0f4b9/users/b414833620494d30a518cb333d39b3d5/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:33.597784 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff7958-cde3-4ce9-9850-d336845fc61e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e26332398f34ada9b7d568f01c0f4b9, user_id=b414833620494d30a518cb333d39b3d5, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:33.600461 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edff7958-cde3-4ce9-9850-d336845fc61e admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:33.662580 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1428/2871] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:23:33 2022] PUT /identity/v3/projects/5e26332398f34ada9b7d568f01c0f4b9/users/b414833620494d30a518cb333d39b3d5/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:33.672563 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b32ad-0042-418f-8f14-899fa70dd568 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:33.672855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b32ad-0042-418f-8f14-899fa70dd568 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:33.673160 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b32ad-0042-418f-8f14-899fa70dd568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:33.701188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-4f1b32ad-0042-418f-8f14-899fa70dd568 None None] MFA Rules not processed for user `b414833620494d30a518cb333d39b3d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:23:33.702423 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-4f1b32ad-0042-418f-8f14-899fa70dd568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:33.771613 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1444/2872] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:23:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:23:33.776197 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff7958-cde3-4ce9-9850-d336845fc61e admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:33.836549 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff7958-cde3-4ce9-9850-d336845fc61e admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:33.841514 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc231bec-b894-456a-9c98-49aa09c97cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:33.842150 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc231bec-b894-456a-9c98-49aa09c97cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:33.842478 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc231bec-b894-456a-9c98-49aa09c97cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:33.842758 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc231bec-b894-456a-9c98-49aa09c97cf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:33.843407 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc231bec-b894-456a-9c98-49aa09c97cf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:33.845303 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc231bec-b894-456a-9c98-49aa09c97cf7 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:33.937176 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1429/2873] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:23:33 2022] POST /identity/v3/projects => generated 367 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:23:33.938216 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b32ad-0042-418f-8f14-899fa70dd568 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:33.948986 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b32ad-0042-418f-8f14-899fa70dd568 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:34.003687 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7e3e4-54d1-49a4-a889-1ecdcace5e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:34.004302 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7e3e4-54d1-49a4-a889-1ecdcace5e13 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:34.004588 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7e3e4-54d1-49a4-a889-1ecdcace5e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:34.004874 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7e3e4-54d1-49a4-a889-1ecdcace5e13 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:34.005540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7e3e4-54d1-49a4-a889-1ecdcace5e13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:34.007507 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7e3e4-54d1-49a4-a889-1ecdcace5e13 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:34.068670 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1445/2874] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:23:33 2022] POST /identity/v3/users => generated 290 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:23:34.072794 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc231bec-b894-456a-9c98-49aa09c97cf7 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:34.084351 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc231bec-b894-456a-9c98-49aa09c97cf7 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:34.090031 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69ee83-9b20-42f6-a623-bb9939b3be96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:34.090648 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69ee83-9b20-42f6-a623-bb9939b3be96 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:34.090933 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69ee83-9b20-42f6-a623-bb9939b3be96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:34.091209 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c69ee83-9b20-42f6-a623-bb9939b3be96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:34.091902 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69ee83-9b20-42f6-a623-bb9939b3be96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:34.093502 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c69ee83-9b20-42f6-a623-bb9939b3be96 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:34.105267 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1430/2875] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:23:34.116133 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7e3e4-54d1-49a4-a889-1ecdcace5e13 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:34.126442 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7e3e4-54d1-49a4-a889-1ecdcace5e13 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:34.131173 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47c7d5-c215-4c57-ac23-27262e540d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:34.131832 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47c7d5-c215-4c57-ac23-27262e540d7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:34.132120 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47c7d5-c215-4c57-ac23-27262e540d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:34.132403 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47c7d5-c215-4c57-ac23-27262e540d7a admin admin] PATH_INFO: `/v3/projects/d33c959ab61447268a9e61e8e2456bf7/users/84825051831a49938b6c60382fda4844/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:34.147079 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47c7d5-c215-4c57-ac23-27262e540d7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d33c959ab61447268a9e61e8e2456bf7, user_id=84825051831a49938b6c60382fda4844, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:34.153361 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47c7d5-c215-4c57-ac23-27262e540d7a admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:34.235070 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1446/2876] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:23:34 2022] PUT /identity/v3/projects/d33c959ab61447268a9e61e8e2456bf7/users/84825051831a49938b6c60382fda4844/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:34.238999 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69ee83-9b20-42f6-a623-bb9939b3be96 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:34.249458 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c69ee83-9b20-42f6-a623-bb9939b3be96 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:34.269483 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:34.307111 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc361cc8-504c-4f97-aa5e-89a52828bc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:34.307716 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc361cc8-504c-4f97-aa5e-89a52828bc07 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:34.307997 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc361cc8-504c-4f97-aa5e-89a52828bc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:34.308272 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc361cc8-504c-4f97-aa5e-89a52828bc07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:34.308954 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc361cc8-504c-4f97-aa5e-89a52828bc07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:34.310570 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc361cc8-504c-4f97-aa5e-89a52828bc07 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:34.320601 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1431/2877] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:23:34.324527 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47c7d5-c215-4c57-ac23-27262e540d7a admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:34.330173 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:34.335856 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47c7d5-c215-4c57-ac23-27262e540d7a admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:34.341128 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f91da-f7a7-4c07-8849-4a8ea6827d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:34.341825 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f91da-f7a7-4c07-8849-4a8ea6827d77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:34.342176 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f91da-f7a7-4c07-8849-4a8ea6827d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:34.342457 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f91da-f7a7-4c07-8849-4a8ea6827d77 admin admin] PATH_INFO: `/v3/projects/d33c959ab61447268a9e61e8e2456bf7/users/84825051831a49938b6c60382fda4844/roles/8ce0850e15804f3d811e88cc9841e673` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:34.344510 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f91da-f7a7-4c07-8849-4a8ea6827d77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d33c959ab61447268a9e61e8e2456bf7, user_id=84825051831a49938b6c60382fda4844, role_id=8ce0850e15804f3d811e88cc9841e673)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:34.347309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f91da-f7a7-4c07-8849-4a8ea6827d77 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:34.358124 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1447/2878] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:23:34 2022] PUT /identity/v3/projects/d33c959ab61447268a9e61e8e2456bf7/users/84825051831a49938b6c60382fda4844/roles/8ce0850e15804f3d811e88cc9841e673 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:34.368105 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb890e-2312-40e2-b3b6-d23358409d73 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:34.368399 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb890e-2312-40e2-b3b6-d23358409d73 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:34.368684 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb890e-2312-40e2-b3b6-d23358409d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:34.396238 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-25bb890e-2312-40e2-b3b6-d23358409d73 None None] MFA Rules not processed for user `84825051831a49938b6c60382fda4844`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:23:34.397444 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-25bb890e-2312-40e2-b3b6-d23358409d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:34.467514 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1432/2879] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:23:34 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:23:34.478999 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f91da-f7a7-4c07-8849-4a8ea6827d77 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:34.480886 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d60f91da-f7a7-4c07-8849-4a8ea6827d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:34.539178 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f91da-f7a7-4c07-8849-4a8ea6827d77 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:34.542773 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45bafe-1c5d-4431-a6b9-9623889111a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:34.543391 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45bafe-1c5d-4431-a6b9-9623889111a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:34.543680 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45bafe-1c5d-4431-a6b9-9623889111a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:34.543956 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45bafe-1c5d-4431-a6b9-9623889111a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:34.553416 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e45bafe-1c5d-4431-a6b9-9623889111a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:34.554873 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e45bafe-1c5d-4431-a6b9-9623889111a5 service cinder] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:34.564454 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1448/2880] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:23:34 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:23:34.756871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb890e-2312-40e2-b3b6-d23358409d73 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:34.767548 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb890e-2312-40e2-b3b6-d23358409d73 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:34.771229 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd492394-cb4f-4186-9ad3-48f12e693ee0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:34.771878 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd492394-cb4f-4186-9ad3-48f12e693ee0 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:34.772162 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd492394-cb4f-4186-9ad3-48f12e693ee0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:34.772438 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd492394-cb4f-4186-9ad3-48f12e693ee0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:34.830508 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd492394-cb4f-4186-9ad3-48f12e693ee0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:34.831871 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd492394-cb4f-4186-9ad3-48f12e693ee0 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:34.840873 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1433/2881] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:23:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:23:40.991336 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45bafe-1c5d-4431-a6b9-9623889111a5 service cinder] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:41.055143 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45bafe-1c5d-4431-a6b9-9623889111a5 service cinder] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:41.059641 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae7154-06c3-4409-ad95-6f0a341d8eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:41.060306 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae7154-06c3-4409-ad95-6f0a341d8eee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:41.060589 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae7154-06c3-4409-ad95-6f0a341d8eee admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:41.060866 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ae7154-06c3-4409-ad95-6f0a341d8eee admin admin] PATH_INFO: `/v3/users/b414833620494d30a518cb333d39b3d5` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:41.062046 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae7154-06c3-4409-ad95-6f0a341d8eee admin admin] RBAC: Authorizing `identity:delete_user(user_id=b414833620494d30a518cb333d39b3d5)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:41.063980 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ae7154-06c3-4409-ad95-6f0a341d8eee admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:41.247309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-98ae7154-06c3-4409-ad95-6f0a341d8eee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b414833620494d30a518cb333d39b3d5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:41.254295 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-98ae7154-06c3-4409-ad95-6f0a341d8eee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b414833620494d30a518cb333d39b3d5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:41.255260 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-98ae7154-06c3-4409-ad95-6f0a341d8eee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b414833620494d30a518cb333d39b3d5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:41.267635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-98ae7154-06c3-4409-ad95-6f0a341d8eee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b414833620494d30a518cb333d39b3d5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:41.268266 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-98ae7154-06c3-4409-ad95-6f0a341d8eee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b414833620494d30a518cb333d39b3d5'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:41.376444 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1449/2882] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:23:40 2022] DELETE /identity/v3/users/b414833620494d30a518cb333d39b3d5 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:41.381006 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce9b28-cec6-46a7-8a5d-9af7ee624059 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:41.381538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce9b28-cec6-46a7-8a5d-9af7ee624059 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:41.381538 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce9b28-cec6-46a7-8a5d-9af7ee624059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:41.400699 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-d8ce9b28-cec6-46a7-8a5d-9af7ee624059 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:23:41.401917 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-d8ce9b28-cec6-46a7-8a5d-9af7ee624059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:41.442011 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=100 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:41.460088 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1434/2883] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:23:41.469783 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae7154-06c3-4409-ad95-6f0a341d8eee admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:41.471188 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-98ae7154-06c3-4409-ad95-6f0a341d8eee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:41.485356 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:41.528476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ae7154-06c3-4409-ad95-6f0a341d8eee admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:41.532141 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bdad41-a2ea-439a-b3a9-6a40d17dca50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:41.532758 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdad41-a2ea-439a-b3a9-6a40d17dca50 service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:41.533041 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdad41-a2ea-439a-b3a9-6a40d17dca50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:41.533322 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bdad41-a2ea-439a-b3a9-6a40d17dca50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:41.544172 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bdad41-a2ea-439a-b3a9-6a40d17dca50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:41.545614 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bdad41-a2ea-439a-b3a9-6a40d17dca50 service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:41.555398 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1450/2884] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:23:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:23:41.944243 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce9b28-cec6-46a7-8a5d-9af7ee624059 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:41.945729 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-d8ce9b28-cec6-46a7-8a5d-9af7ee624059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:41.959751 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce9b28-cec6-46a7-8a5d-9af7ee624059 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:41.964181 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438d552-a4b1-42b0-b20f-c5d13b03ea42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:41.964734 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438d552-a4b1-42b0-b20f-c5d13b03ea42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:41.965035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438d552-a4b1-42b0-b20f-c5d13b03ea42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:41.965310 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438d552-a4b1-42b0-b20f-c5d13b03ea42 admin admin] PATH_INFO: `/v3/projects/5e26332398f34ada9b7d568f01c0f4b9` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:41.966569 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438d552-a4b1-42b0-b20f-c5d13b03ea42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e26332398f34ada9b7d568f01c0f4b9)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:41.968504 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438d552-a4b1-42b0-b20f-c5d13b03ea42 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:42.051223 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e438d552-a4b1-42b0-b20f-c5d13b03ea42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e26332398f34ada9b7d568f01c0f4b9', 'request_id': 'req-e438d552-a4b1-42b0-b20f-c5d13b03ea42'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:42.057782 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e438d552-a4b1-42b0-b20f-c5d13b03ea42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e26332398f34ada9b7d568f01c0f4b9', 'request_id': 'req-e438d552-a4b1-42b0-b20f-c5d13b03ea42'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:42.063299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e438d552-a4b1-42b0-b20f-c5d13b03ea42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e26332398f34ada9b7d568f01c0f4b9', 'request_id': 'req-e438d552-a4b1-42b0-b20f-c5d13b03ea42'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:42.118333 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e438d552-a4b1-42b0-b20f-c5d13b03ea42 admin admin] The token cache is being invalidate because project 5e26332398f34ada9b7d568f01c0f4b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:23:42.118624 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e438d552-a4b1-42b0-b20f-c5d13b03ea42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:42.119251 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e438d552-a4b1-42b0-b20f-c5d13b03ea42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:42.120909 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1435/2885] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:23:41 2022] DELETE /identity/v3/projects/5e26332398f34ada9b7d568f01c0f4b9 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:42.124993 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bdad41-a2ea-439a-b3a9-6a40d17dca50 service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:42.135193 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bdad41-a2ea-439a-b3a9-6a40d17dca50 service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:42.190981 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621f63f0-d00c-4ad8-a650-27f4eb5b14a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:42.191618 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f63f0-d00c-4ad8-a650-27f4eb5b14a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:42.191902 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f63f0-d00c-4ad8-a650-27f4eb5b14a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:42.192179 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621f63f0-d00c-4ad8-a650-27f4eb5b14a0 admin admin] PATH_INFO: `/v3/users/84825051831a49938b6c60382fda4844` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:42.193330 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f63f0-d00c-4ad8-a650-27f4eb5b14a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84825051831a49938b6c60382fda4844)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:42.195386 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621f63f0-d00c-4ad8-a650-27f4eb5b14a0 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:42.401431 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-621f63f0-d00c-4ad8-a650-27f4eb5b14a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84825051831a49938b6c60382fda4844'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:42.408179 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-621f63f0-d00c-4ad8-a650-27f4eb5b14a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84825051831a49938b6c60382fda4844'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:42.408846 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-621f63f0-d00c-4ad8-a650-27f4eb5b14a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84825051831a49938b6c60382fda4844'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:42.420539 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-621f63f0-d00c-4ad8-a650-27f4eb5b14a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84825051831a49938b6c60382fda4844'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:42.421195 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-621f63f0-d00c-4ad8-a650-27f4eb5b14a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84825051831a49938b6c60382fda4844'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:42.507320 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1451/2886] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:23:42 2022] DELETE /identity/v3/users/84825051831a49938b6c60382fda4844 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:42.892533 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438d552-a4b1-42b0-b20f-c5d13b03ea42 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:42.893992 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e438d552-a4b1-42b0-b20f-c5d13b03ea42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:42.909347 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438d552-a4b1-42b0-b20f-c5d13b03ea42 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:42.968995 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01d584-ed97-4318-a2dd-08ef578dc868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:42.969650 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01d584-ed97-4318-a2dd-08ef578dc868 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:42.969955 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01d584-ed97-4318-a2dd-08ef578dc868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:42.970242 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01d584-ed97-4318-a2dd-08ef578dc868 admin admin] PATH_INFO: `/v3/projects/d33c959ab61447268a9e61e8e2456bf7` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:42.971404 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01d584-ed97-4318-a2dd-08ef578dc868 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d33c959ab61447268a9e61e8e2456bf7)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:42.973312 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01d584-ed97-4318-a2dd-08ef578dc868 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:43.073788 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8c01d584-ed97-4318-a2dd-08ef578dc868 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd33c959ab61447268a9e61e8e2456bf7', 'request_id': 'req-8c01d584-ed97-4318-a2dd-08ef578dc868'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:43.079850 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8c01d584-ed97-4318-a2dd-08ef578dc868 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd33c959ab61447268a9e61e8e2456bf7', 'request_id': 'req-8c01d584-ed97-4318-a2dd-08ef578dc868'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:43.085360 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8c01d584-ed97-4318-a2dd-08ef578dc868 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd33c959ab61447268a9e61e8e2456bf7', 'request_id': 'req-8c01d584-ed97-4318-a2dd-08ef578dc868'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:43.131918 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:43.136622 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8c01d584-ed97-4318-a2dd-08ef578dc868 admin admin] The token cache is being invalidate because project d33c959ab61447268a9e61e8e2456bf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:23:43.148883 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8c01d584-ed97-4318-a2dd-08ef578dc868 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:43.148883 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-8c01d584-ed97-4318-a2dd-08ef578dc868 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:23:43.148883 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1436/2887] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:23:42 2022] DELETE /identity/v3/projects/d33c959ab61447268a9e61e8e2456bf7 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:43.149207 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4e2bc-f3d9-413b-9eef-c6bb6a197f5a None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:43.149207 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4e2bc-f3d9-413b-9eef-c6bb6a197f5a None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:43.149207 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4e2bc-f3d9-413b-9eef-c6bb6a197f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:43.164365 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-3aa4e2bc-f3d9-413b-9eef-c6bb6a197f5a None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:23:43.166248 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-3aa4e2bc-f3d9-413b-9eef-c6bb6a197f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:43.194866 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:43.224635 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1452/2888] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 09:23:43.233606 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1f4d8-2773-4258-ae03-c4287af530e3 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:43.233905 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1f4d8-2773-4258-ae03-c4287af530e3 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:43.234207 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b1f4d8-2773-4258-ae03-c4287af530e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:43.253783 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-77b1f4d8-2773-4258-ae03-c4287af530e3 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:23:43.255030 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-77b1f4d8-2773-4258-ae03-c4287af530e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:43.263800 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1437/2889] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:23:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:23:43.268095 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4e2bc-f3d9-413b-9eef-c6bb6a197f5a None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:43.278583 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4e2bc-f3d9-413b-9eef-c6bb6a197f5a None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:43.283712 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17973033-0dfb-440e-8cc2-f9349287c5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:43.284321 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17973033-0dfb-440e-8cc2-f9349287c5fb admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:43.284602 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17973033-0dfb-440e-8cc2-f9349287c5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:43.284878 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17973033-0dfb-440e-8cc2-f9349287c5fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:43.285569 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17973033-0dfb-440e-8cc2-f9349287c5fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:23:43.285869 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17973033-0dfb-440e-8cc2-f9349287c5fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:43.287485 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17973033-0dfb-440e-8cc2-f9349287c5fb admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:43.299410 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1453/2890] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 09:23:43.303960 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1f4d8-2773-4258-ae03-c4287af530e3 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:43.313403 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b1f4d8-2773-4258-ae03-c4287af530e3 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:43.318147 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91921c4d-07cf-4943-8b1a-7981bae79158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:43.318793 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91921c4d-07cf-4943-8b1a-7981bae79158 admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:43.319078 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91921c4d-07cf-4943-8b1a-7981bae79158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:43.319356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91921c4d-07cf-4943-8b1a-7981bae79158 admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:43.320035 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91921c4d-07cf-4943-8b1a-7981bae79158 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:43.322352 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91921c4d-07cf-4943-8b1a-7981bae79158 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:43.383153 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1438/2891] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:23:43 2022] POST /identity/v3/projects => generated 377 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:23:43.387324 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17973033-0dfb-440e-8cc2-f9349287c5fb admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:43.397270 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17973033-0dfb-440e-8cc2-f9349287c5fb admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:43.452673 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed6388-0382-41f7-8c57-9262068a2d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:43.453216 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed6388-0382-41f7-8c57-9262068a2d84 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:43.453499 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed6388-0382-41f7-8c57-9262068a2d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:43.453776 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed6388-0382-41f7-8c57-9262068a2d84 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:43.454474 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed6388-0382-41f7-8c57-9262068a2d84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:43.456374 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed6388-0382-41f7-8c57-9262068a2d84 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:43.513512 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1454/2892] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:23:43 2022] POST /identity/v3/users => generated 289 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:23:43.517934 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91921c4d-07cf-4943-8b1a-7981bae79158 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:43.529449 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91921c4d-07cf-4943-8b1a-7981bae79158 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:43.534298 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947357e-293d-49bc-a006-9cca15decdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:43.534903 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947357e-293d-49bc-a006-9cca15decdb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:43.535197 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947357e-293d-49bc-a006-9cca15decdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:43.535457 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5947357e-293d-49bc-a006-9cca15decdb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:43.536139 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947357e-293d-49bc-a006-9cca15decdb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:43.537753 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5947357e-293d-49bc-a006-9cca15decdb3 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:43.547620 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1439/2893] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:23:43.551649 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed6388-0382-41f7-8c57-9262068a2d84 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:43.562410 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed6388-0382-41f7-8c57-9262068a2d84 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:43.567088 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723e0c8-2249-4545-a7f7-261aa9e0ec0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:43.567696 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723e0c8-2249-4545-a7f7-261aa9e0ec0d admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:43.567983 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723e0c8-2249-4545-a7f7-261aa9e0ec0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:43.568262 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723e0c8-2249-4545-a7f7-261aa9e0ec0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:43.568955 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723e0c8-2249-4545-a7f7-261aa9e0ec0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:43.570582 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723e0c8-2249-4545-a7f7-261aa9e0ec0d admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:43.581548 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1455/2894] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:23:43.585515 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947357e-293d-49bc-a006-9cca15decdb3 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:43.596017 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5947357e-293d-49bc-a006-9cca15decdb3 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:43.600653 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5848fc-38a2-45d9-a25b-6dc3da95d45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:43.601299 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5848fc-38a2-45d9-a25b-6dc3da95d45b admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:43.601581 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5848fc-38a2-45d9-a25b-6dc3da95d45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:43.601859 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5848fc-38a2-45d9-a25b-6dc3da95d45b admin admin] PATH_INFO: `/v3/projects/20455c9c727f49709f306c46788cf590/users/72fb7b3c370d40f1a27ee186ec66d2d4/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:43.617102 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5848fc-38a2-45d9-a25b-6dc3da95d45b admin admin] RBAC: Authorizing `identity:create_grant(project_id=20455c9c727f49709f306c46788cf590, user_id=72fb7b3c370d40f1a27ee186ec66d2d4, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:43.619785 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5848fc-38a2-45d9-a25b-6dc3da95d45b admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:43.690750 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1440/2895] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:23:43 2022] PUT /identity/v3/projects/20455c9c727f49709f306c46788cf590/users/72fb7b3c370d40f1a27ee186ec66d2d4/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:23:43.701582 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684cf2e-d1e2-4e2c-aab5-9d149e3f60d4 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:43.701872 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684cf2e-d1e2-4e2c-aab5-9d149e3f60d4 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:43.702182 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9684cf2e-d1e2-4e2c-aab5-9d149e3f60d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:43.729792 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-9684cf2e-d1e2-4e2c-aab5-9d149e3f60d4 None None] MFA Rules not processed for user `72fb7b3c370d40f1a27ee186ec66d2d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:23:43.730963 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-9684cf2e-d1e2-4e2c-aab5-9d149e3f60d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:43.792861 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1456/2896] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:23:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:23:43.804672 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5848fc-38a2-45d9-a25b-6dc3da95d45b admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:23:43.806897 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-6f5848fc-38a2-45d9-a25b-6dc3da95d45b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:23:43.884811 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5848fc-38a2-45d9-a25b-6dc3da95d45b admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:23:43.888435 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2bfd8-e06f-4ab7-a6b2-1d02a3c278d7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4abd1512ad224da896c6f2257f9602e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:23:43.889041 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2bfd8-e06f-4ab7-a6b2-1d02a3c278d7 service glance] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:23:43.889318 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2bfd8-e06f-4ab7-a6b2-1d02a3c278d7 service glance] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:23:43.889593 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2bfd8-e06f-4ab7-a6b2-1d02a3c278d7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:23:43.897638 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2bfd8-e06f-4ab7-a6b2-1d02a3c278d7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:23:43.899089 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2bfd8-e06f-4ab7-a6b2-1d02a3c278d7 service glance] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:23:43.907703 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1441/2897] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:23:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:23:53.789996 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:23:53.903751 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=63,keystone:INSERT=2 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:24:06.264262 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684cf2e-d1e2-4e2c-aab5-9d149e3f60d4 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:24:06.328623 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9684cf2e-d1e2-4e2c-aab5-9d149e3f60d4 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:24:06.334339 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195d7c1-aa62-4a77-931c-9c7dc0e472d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:24:06.334984 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195d7c1-aa62-4a77-931c-9c7dc0e472d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:24:06.335263 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195d7c1-aa62-4a77-931c-9c7dc0e472d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:24:06.335537 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195d7c1-aa62-4a77-931c-9c7dc0e472d0 admin admin] PATH_INFO: `/v3/users/72fb7b3c370d40f1a27ee186ec66d2d4` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:24:06.336689 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1195d7c1-aa62-4a77-931c-9c7dc0e472d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72fb7b3c370d40f1a27ee186ec66d2d4)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:24:06.338724 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1195d7c1-aa62-4a77-931c-9c7dc0e472d0 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:24:06.543226 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1195d7c1-aa62-4a77-931c-9c7dc0e472d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72fb7b3c370d40f1a27ee186ec66d2d4'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:24:06.549941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1195d7c1-aa62-4a77-931c-9c7dc0e472d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72fb7b3c370d40f1a27ee186ec66d2d4'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:24:06.550604 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1195d7c1-aa62-4a77-931c-9c7dc0e472d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72fb7b3c370d40f1a27ee186ec66d2d4'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:24:06.562978 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1195d7c1-aa62-4a77-931c-9c7dc0e472d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72fb7b3c370d40f1a27ee186ec66d2d4'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:24:06.563606 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-1195d7c1-aa62-4a77-931c-9c7dc0e472d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72fb7b3c370d40f1a27ee186ec66d2d4'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:24:06.655672 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1457/2898] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:24:06 2022] DELETE /identity/v3/users/72fb7b3c370d40f1a27ee186ec66d2d4 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 09:24:06.660443 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d2d4c-9e69-4909-a657-db962a3de043 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:24:06.660956 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d2d4c-9e69-4909-a657-db962a3de043 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:24:06.661073 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3d2d4c-9e69-4909-a657-db962a3de043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:24:06.681719 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-4c3d2d4c-9e69-4909-a657-db962a3de043 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:24:06.683565 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4c3d2d4c-9e69-4909-a657-db962a3de043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:24:06.741898 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1442/2899] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:24:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:24:06.752560 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195d7c1-aa62-4a77-931c-9c7dc0e472d0 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:24:06.754004 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-1195d7c1-aa62-4a77-931c-9c7dc0e472d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:24:06.811515 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195d7c1-aa62-4a77-931c-9c7dc0e472d0 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:24:06.815092 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ba4e92-62b4-4356-a306-698552aab2fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:24:06.815716 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba4e92-62b4-4356-a306-698552aab2fe service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:24:06.815997 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba4e92-62b4-4356-a306-698552aab2fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:24:06.816296 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ba4e92-62b4-4356-a306-698552aab2fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:24:06.825112 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba4e92-62b4-4356-a306-698552aab2fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:24:06.826571 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ba4e92-62b4-4356-a306-698552aab2fe service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:24:06.835765 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1458/2900] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:24:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:24:07.257224 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d2d4c-9e69-4909-a657-db962a3de043 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:24:07.258681 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-4c3d2d4c-9e69-4909-a657-db962a3de043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:24:07.271385 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3d2d4c-9e69-4909-a657-db962a3de043 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:24:07.277500 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bbdd63-33b0-4ed7-8bd5-b16fce06aa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:24:07.278599 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbdd63-33b0-4ed7-8bd5-b16fce06aa89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:24:07.278885 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbdd63-33b0-4ed7-8bd5-b16fce06aa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:24:07.279157 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbdd63-33b0-4ed7-8bd5-b16fce06aa89 admin admin] PATH_INFO: `/v3/projects/20455c9c727f49709f306c46788cf590` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:24:07.280337 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bbdd63-33b0-4ed7-8bd5-b16fce06aa89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20455c9c727f49709f306c46788cf590)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:24:07.283551 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bbdd63-33b0-4ed7-8bd5-b16fce06aa89 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:24:07.375514 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e9bbdd63-33b0-4ed7-8bd5-b16fce06aa89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20455c9c727f49709f306c46788cf590', 'request_id': 'req-e9bbdd63-33b0-4ed7-8bd5-b16fce06aa89'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:24:07.381540 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e9bbdd63-33b0-4ed7-8bd5-b16fce06aa89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20455c9c727f49709f306c46788cf590', 'request_id': 'req-e9bbdd63-33b0-4ed7-8bd5-b16fce06aa89'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:24:07.387567 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e9bbdd63-33b0-4ed7-8bd5-b16fce06aa89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20455c9c727f49709f306c46788cf590', 'request_id': 'req-e9bbdd63-33b0-4ed7-8bd5-b16fce06aa89'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:24:07.460584 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e9bbdd63-33b0-4ed7-8bd5-b16fce06aa89 admin admin] The token cache is being invalidate because project 20455c9c727f49709f306c46788cf590 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:24:07.460876 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e9bbdd63-33b0-4ed7-8bd5-b16fce06aa89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:24:07.461506 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-e9bbdd63-33b0-4ed7-8bd5-b16fce06aa89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:24:07.463683 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1443/2901] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:24:07 2022] DELETE /identity/v3/projects/20455c9c727f49709f306c46788cf590 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:24:07.468589 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e196a-d5ac-4585-a6f9-7085920ad30e None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:24:07.469077 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e196a-d5ac-4585-a6f9-7085920ad30e None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:24:07.469195 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e196a-d5ac-4585-a6f9-7085920ad30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:24:07.488591 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-f22e196a-d5ac-4585-a6f9-7085920ad30e None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:24:07.489839 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f22e196a-d5ac-4585-a6f9-7085920ad30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:24:07.506282 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:24:07.547838 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1459/2902] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:24:07.557034 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955d9d8-9165-4fef-8f7a-a3635a7bf3c8 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:24:07.557356 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955d9d8-9165-4fef-8f7a-a3635a7bf3c8 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:24:07.557638 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955d9d8-9165-4fef-8f7a-a3635a7bf3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:24:07.577951 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-2955d9d8-9165-4fef-8f7a-a3635a7bf3c8 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:24:07.579165 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-2955d9d8-9165-4fef-8f7a-a3635a7bf3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:24:07.587998 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1444/2903] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:24:07.592941 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e196a-d5ac-4585-a6f9-7085920ad30e None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:24:07.602984 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e196a-d5ac-4585-a6f9-7085920ad30e None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:24:07.607524 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4441b6d2-4046-4f79-b566-67047358ee49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:24:07.608126 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441b6d2-4046-4f79-b566-67047358ee49 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:24:07.608405 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441b6d2-4046-4f79-b566-67047358ee49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:24:07.608679 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441b6d2-4046-4f79-b566-67047358ee49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:24:07.609367 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4441b6d2-4046-4f79-b566-67047358ee49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80328) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 09:24:07.609668 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4441b6d2-4046-4f79-b566-67047358ee49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:24:07.611292 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4441b6d2-4046-4f79-b566-67047358ee49 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:24:07.624678 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1460/2904] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 2 09:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 09:24:07.629201 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955d9d8-9165-4fef-8f7a-a3635a7bf3c8 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:24:07.639051 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955d9d8-9165-4fef-8f7a-a3635a7bf3c8 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:24:07.643731 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7797e96-1edc-4853-b9c9-01c29710bfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:24:07.644332 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7797e96-1edc-4853-b9c9-01c29710bfae admin admin] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:24:07.644617 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7797e96-1edc-4853-b9c9-01c29710bfae admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:24:07.644893 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7797e96-1edc-4853-b9c9-01c29710bfae admin admin] PATH_INFO: `/v3/projects` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:24:07.645559 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7797e96-1edc-4853-b9c9-01c29710bfae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:24:07.647537 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7797e96-1edc-4853-b9c9-01c29710bfae admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:24:07.777551 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1445/2905] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 2 09:24:07 2022] POST /identity/v3/projects => generated 385 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 09:24:07.781694 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4441b6d2-4046-4f79-b566-67047358ee49 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:24:07.791711 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4441b6d2-4046-4f79-b566-67047358ee49 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:24:07.846933 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d04c4-0547-409a-9c4a-bef7ea693b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:24:07.847543 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d04c4-0547-409a-9c4a-bef7ea693b89 admin admin] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:24:07.847822 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d04c4-0547-409a-9c4a-bef7ea693b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:24:07.848095 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d04c4-0547-409a-9c4a-bef7ea693b89 admin admin] PATH_INFO: `/v3/users` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:24:07.848755 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d04c4-0547-409a-9c4a-bef7ea693b89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:24:07.850680 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52d04c4-0547-409a-9c4a-bef7ea693b89 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:24:07.924721 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1461/2906] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 2 09:24:07 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 09:24:07.929105 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7797e96-1edc-4853-b9c9-01c29710bfae admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:24:07.940616 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7797e96-1edc-4853-b9c9-01c29710bfae admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:24:07.945436 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed661e-09cd-40bf-a20f-477c9981dc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:24:07.946148 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed661e-09cd-40bf-a20f-477c9981dc1d admin admin] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:24:07.946433 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed661e-09cd-40bf-a20f-477c9981dc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:24:07.952059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed661e-09cd-40bf-a20f-477c9981dc1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:24:07.952059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed661e-09cd-40bf-a20f-477c9981dc1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:24:07.952059 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed661e-09cd-40bf-a20f-477c9981dc1d admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:24:07.960375 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1446/2907] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:24:07.964632 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d04c4-0547-409a-9c4a-bef7ea693b89 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:24:07.974644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52d04c4-0547-409a-9c4a-bef7ea693b89 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:24:07.979304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28042428-620d-4f06-8fd4-2e6b6f062862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:24:07.980742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28042428-620d-4f06-8fd4-2e6b6f062862 admin admin] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:24:07.980742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28042428-620d-4f06-8fd4-2e6b6f062862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:24:07.980742 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28042428-620d-4f06-8fd4-2e6b6f062862 admin admin] PATH_INFO: `/v3/roles` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:24:07.981175 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28042428-620d-4f06-8fd4-2e6b6f062862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:24:07.982796 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28042428-620d-4f06-8fd4-2e6b6f062862 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:24:07.993644 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1462/2908] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 2 09:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 09:24:07.998144 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed661e-09cd-40bf-a20f-477c9981dc1d admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:24:08.007944 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed661e-09cd-40bf-a20f-477c9981dc1d admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:24:08.013102 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93fb099-0b44-482a-8cce-5d72e2e87cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:24:08.013746 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fb099-0b44-482a-8cce-5d72e2e87cb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:24:08.014067 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fb099-0b44-482a-8cce-5d72e2e87cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:24:08.014343 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93fb099-0b44-482a-8cce-5d72e2e87cb8 admin admin] PATH_INFO: `/v3/projects/7cc901e8933c4a4b8b3c3d5f903cd541/users/fee2fe1544d4419687a5be64034b20b1/roles/68fe920c9be84b3aa1d35bdce554d96a` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:24:08.028444 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93fb099-0b44-482a-8cce-5d72e2e87cb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cc901e8933c4a4b8b3c3d5f903cd541, user_id=fee2fe1544d4419687a5be64034b20b1, role_id=68fe920c9be84b3aa1d35bdce554d96a)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:24:08.031108 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93fb099-0b44-482a-8cce-5d72e2e87cb8 admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:24:08.113021 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1447/2909] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 2 09:24:07 2022] PUT /identity/v3/projects/7cc901e8933c4a4b8b3c3d5f903cd541/users/fee2fe1544d4419687a5be64034b20b1/roles/68fe920c9be84b3aa1d35bdce554d96a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:24:08.123814 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65bd07-6803-4b33-8225-ba126e2bfaf1 None None] REQUEST_METHOD: `POST` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:24:08.124099 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65bd07-6803-4b33-8225-ba126e2bfaf1 None None] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:24:08.124375 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65bd07-6803-4b33-8225-ba126e2bfaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:24:08.151926 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.auth.core [None req-6b65bd07-6803-4b33-8225-ba126e2bfaf1 None None] MFA Rules not processed for user `fee2fe1544d4419687a5be64034b20b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80328) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:24:08.153113 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6b65bd07-6803-4b33-8225-ba126e2bfaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:24:08.216423 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1463/2910] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:24:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:24:08.227728 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93fb099-0b44-482a-8cce-5d72e2e87cb8 admin admin] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:24:08.229935 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-e93fb099-0b44-482a-8cce-5d72e2e87cb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:24:08.252390 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:24:08.287375 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93fb099-0b44-482a-8cce-5d72e2e87cb8 admin admin] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:24:08.292047 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f4bcd-2401-4b2b-8c84-6552dcba5174 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6e39392b79bc439780415db7c8206098', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:24:08.292660 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f4bcd-2401-4b2b-8c84-6552dcba5174 service cinder] REQUEST_METHOD: `GET` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:24:08.292945 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f4bcd-2401-4b2b-8c84-6552dcba5174 service cinder] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:24:08.293236 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f4bcd-2401-4b2b-8c84-6552dcba5174 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:24:08.301528 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f4bcd-2401-4b2b-8c84-6552dcba5174 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:24:08.303211 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f4bcd-2401-4b2b-8c84-6552dcba5174 service cinder] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:24:08.313029 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1448/2911] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:24:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:24:12.395476 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65bd07-6803-4b33-8225-ba126e2bfaf1 None None] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:24:12.414705 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:24:12.457205 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65bd07-6803-4b33-8225-ba126e2bfaf1 None None] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:24:12.461777 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa84d95-7377-484a-9698-d71297fa9008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:24:12.462442 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa84d95-7377-484a-9698-d71297fa9008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:24:12.462727 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa84d95-7377-484a-9698-d71297fa9008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:24:12.463008 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa84d95-7377-484a-9698-d71297fa9008 admin admin] PATH_INFO: `/v3/users/fee2fe1544d4419687a5be64034b20b1` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:24:12.464677 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa84d95-7377-484a-9698-d71297fa9008 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fee2fe1544d4419687a5be64034b20b1)` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:24:12.466640 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa84d95-7377-484a-9698-d71297fa9008 admin admin] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:24:12.639645 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6aa84d95-7377-484a-9698-d71297fa9008 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fee2fe1544d4419687a5be64034b20b1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:24:12.646317 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6aa84d95-7377-484a-9698-d71297fa9008 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fee2fe1544d4419687a5be64034b20b1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:24:12.649323 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6aa84d95-7377-484a-9698-d71297fa9008 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fee2fe1544d4419687a5be64034b20b1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:24:12.659481 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6aa84d95-7377-484a-9698-d71297fa9008 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fee2fe1544d4419687a5be64034b20b1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:24:12.660157 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.notifications [None req-6aa84d95-7377-484a-9698-d71297fa9008 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fee2fe1544d4419687a5be64034b20b1'} {{(pid=80328) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:24:12.719147 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1464/2912] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 2 09:24:12 2022] DELETE /identity/v3/users/fee2fe1544d4419687a5be64034b20b1 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:24:12.723661 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba65a81-85f5-4c80-814f-ce9afe3c37b1 None None] REQUEST_METHOD: `POST` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:24:12.724146 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba65a81-85f5-4c80-814f-ce9afe3c37b1 None None] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:24:12.726754 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba65a81-85f5-4c80-814f-ce9afe3c37b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:24:12.743016 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.auth.core [None req-bba65a81-85f5-4c80-814f-ce9afe3c37b1 None None] MFA Rules not processed for user `5b34698a764b44aba3d550dca381a370`. Rule list: `[]` (Enabled: `True`). {{(pid=80329) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 09:24:12.744190 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-bba65a81-85f5-4c80-814f-ce9afe3c37b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:24:12.802881 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1449/2913] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 2 09:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 09:24:12.812565 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa84d95-7377-484a-9698-d71297fa9008 admin admin] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:24:12.814023 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-6aa84d95-7377-484a-9698-d71297fa9008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:24:12.869304 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa84d95-7377-484a-9698-d71297fa9008 admin admin] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:24:12.872964 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80dbcff-88e6-4ded-b393-21f913741cae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ddf6f4d84bb34d2f810a0f6cec36da4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:24:12.873570 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80dbcff-88e6-4ded-b393-21f913741cae service neutron] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:24:12.873849 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80dbcff-88e6-4ded-b393-21f913741cae service neutron] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:24:12.874145 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80dbcff-88e6-4ded-b393-21f913741cae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:24:12.882846 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80dbcff-88e6-4ded-b393-21f913741cae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:24:12.884257 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80dbcff-88e6-4ded-b393-21f913741cae service neutron] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:24:12.893946 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1465/2914] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:24:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:24:13.303240 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba65a81-85f5-4c80-814f-ce9afe3c37b1 None None] Authenticating user token {{(pid=80329) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:24:13.305647 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.fernet_utils [None req-bba65a81-85f5-4c80-814f-ce9afe3c37b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80329) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:24:13.317643 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba65a81-85f5-4c80-814f-ce9afe3c37b1 None None] Validating token access rules against request {{(pid=80329) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:24:13.322369 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b5517-a533-474a-bfb6-1e4accb55d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b34698a764b44aba3d550dca381a370', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a92b68d350c4f7998c221772fb56b4a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80329) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:24:13.323126 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b5517-a533-474a-bfb6-1e4accb55d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:24:13.323979 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b5517-a533-474a-bfb6-1e4accb55d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:24:13.324263 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b5517-a533-474a-bfb6-1e4accb55d9c admin admin] PATH_INFO: `/v3/projects/7cc901e8933c4a4b8b3c3d5f903cd541` {{(pid=80329) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:24:13.325472 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b5517-a533-474a-bfb6-1e4accb55d9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cc901e8933c4a4b8b3c3d5f903cd541)` {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:24:13.327443 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b5517-a533-474a-bfb6-1e4accb55d9c admin admin] RBAC: Authorization granted {{(pid=80329) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:24:13.415534 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-841b5517-a533-474a-bfb6-1e4accb55d9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cc901e8933c4a4b8b3c3d5f903cd541', 'request_id': 'req-841b5517-a533-474a-bfb6-1e4accb55d9c'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:24:13.422120 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-841b5517-a533-474a-bfb6-1e4accb55d9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cc901e8933c4a4b8b3c3d5f903cd541', 'request_id': 'req-841b5517-a533-474a-bfb6-1e4accb55d9c'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:24:13.427798 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-841b5517-a533-474a-bfb6-1e4accb55d9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cc901e8933c4a4b8b3c3d5f903cd541', 'request_id': 'req-841b5517-a533-474a-bfb6-1e4accb55d9c'} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:24:13.504867 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-841b5517-a533-474a-bfb6-1e4accb55d9c admin admin] The token cache is being invalidate because project 7cc901e8933c4a4b8b3c3d5f903cd541 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80329) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 09:24:13.505169 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-841b5517-a533-474a-bfb6-1e4accb55d9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:24:13.505823 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG keystone.notifications [None req-841b5517-a533-474a-bfb6-1e4accb55d9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80329) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 09:24:13.507767 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: [pid: 80329|app: 0|req: 1450/2915] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 2 09:24:13 2022] DELETE /identity/v3/projects/7cc901e8933c4a4b8b3c3d5f903cd541 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 09:24:22.889855 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:24:23.499088 dv-cinder-847131-4-iscsi devstack@keystone.service[80329]: DEBUG dbcounter [-] [80329] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80329) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 09:25:25.187638 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80dbcff-88e6-4ded-b393-21f913741cae service neutron] Authenticating user token {{(pid=80328) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 09:25:25.189026 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-f80dbcff-88e6-4ded-b393-21f913741cae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:25:25.278283 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80dbcff-88e6-4ded-b393-21f913741cae service neutron] Validating token access rules against request {{(pid=80328) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 09:25:25.336365 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8267669-db9e-442e-9cf1-7ca6ee33ad99 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cad5cc9927374836b0cc74db85b686fd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4227a5b804c0414c99d407bb1415cf49', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80328) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 09:25:25.336997 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8267669-db9e-442e-9cf1-7ca6ee33ad99 service placement] REQUEST_METHOD: `GET` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 09:25:25.337264 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8267669-db9e-442e-9cf1-7ca6ee33ad99 service placement] SCRIPT_NAME: `/identity` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 09:25:25.337540 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8267669-db9e-442e-9cf1-7ca6ee33ad99 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80328) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 09:25:25.339309 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.fernet_utils [None req-d8267669-db9e-442e-9cf1-7ca6ee33ad99 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80328) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 09:25:25.351053 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8267669-db9e-442e-9cf1-7ca6ee33ad99 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 09:25:25.352652 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8267669-db9e-442e-9cf1-7ca6ee33ad99 service placement] RBAC: Authorization granted {{(pid=80328) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 09:25:25.362482 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: [pid: 80328|app: 0|req: 1466/2916] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 2 09:25:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 09:25:35.358161 dv-cinder-847131-4-iscsi devstack@keystone.service[80328]: DEBUG dbcounter [-] [80328] Writing DB stats keystone:SELECT=67 {{(pid=80328) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}